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Summary 

In  1995,  the  Massachusetts  Legislature  enacted  Chapter  263  of  the  Acts  of  1995, 
known  as  the  "Judiciary  Information  Technology  Loan,  Act  of  1995  (bond  bill)."1  The 
bond  bill  authorized  the  expenditure  of  $75,000,000  for  the  "acquisition,  upgrading, 
development  and  implementation  of  a  comprehensive  integrated  information  system 
within  and  between  the  departments  of  the  trial  court  of  the  commonwealth...."  Due  to 
the  scope  and  complexity  of  the  project,  the  Legislature  instituted  a  series  of  measures 
intended  to  monitor  the  expenditure  of  funds  and  the  progress  of  the  project. 

Notwithstanding  the  stringent  safeguards  imposed  by  the  Legislature  for  the 
monitoring  of  public  assets,  the  Administrative  Office  of  the  Trial  Court  (AOTC)  has 
failed  to  adhere  to  the  planning,  reporting  and  management  requirements  imposed  by 
bond  bill.    While  approxim; 
Bureau  has  determined  that: 


bond  bill.    While  approximately  $40  million  has  been  expended  pursuant  to  the  act,  the 


>  The  AOTC  failed  to  follow  its  Information  Technology  (IT)  Plan 
in  violation  of  St.1992,  c.133,  §37. 

>  The  AOTC  failed  to  file  schedules  of  expenditures  with  the 
Legislature  in  violation  of  the  provisions  of  the  St.  1995,  c.  263; 

>  The  AOTC  failed  to  file  yearly  accountings  and  assessments  with 
the  Legislature  in  violation  of  the  provisions  of  the  St.1995,  c. 
263; 

>  The  AOTC  failed  to  file  yearly  spending  plans  with  the 
Legislature  in  violation  of  the  provisions  of  the  St.1995,  c.  263; 

>  The  AOTC  failed  to  submit  an  adequate  accounting  of  costs 
related  to  the  administration  of  the  project  in  violation  of  the 
provisions  of  St.1995,  c.  263; 

>  The  AOTC  failed  to  inform  the  Legislature  that  it  was  deviating 
from  the  1995  IT  Plan  as  a  result  of  diminished  bond  fund 
allocations. 

All  of  the  above  are  requirements  that  were  specifically  enumerated  within  the 
bond  bill.  Notwithstanding  the  AOTC's  knowledge  of  these  specific  reporting 
requirements,  the  AOTC  has  failed  to  comply  with  the  statutory  prerequisites  established 
within  the  bond  bill. 


1  The  bond  bill  was  amended  in  1996  by  St.  1996,  c.  186  §  3. 

2  The  bond  bill  required  all  plans,  schedules  and  reports  to  be  submitted  to  the  house  and  senate  committees 
on  ways  and  means,  the  house  and  senate  committees  on  science  and  technology,  the  joint  committee  on 
state  administration  and  the  secretary  of  administration  and  finance. 

3  The  amount  referenced  is  based  documents  submitted  by  the  AOTC  to  the  Legislature  on  September  5, 
2001. 


The  Bureau  finds  that  this  failure  to  adhere  to  statutory  directives  is  a  function  of 
a  management  structure  that  is,  at  present,  unable  to  procure,  select,  design,  develop,  and 
implement  the  "comprehensive  integrated  information  system"  envisioned  in  the  bond 
bill.  The  AOTC  lacks  a  coherent  management  philosophy  and  is  presently  unable  to 
effectively  manage  the  internal  AOTC  office;  unable  to  manage  the  individual 
departments  of  the  Trial  Court4  relative  to  the  IT  Project;  and,  unable  to  manage  the  IT 
Project.  The  Bureau  specifically  found  that: 

►  The  AOTC  did  not  hire  a  project  manager  until  the  Spring  of  2000; 

►  The  AOTC  failed  to  adhere  to  the  management  procedures  and 
policies  established  for  the  project; 

►  The  AOTC  failed  to  follow  the  recommendations  of  project 
advisory  and  oversight  committees; 

►  The  AOTC  failed  to  independently  verify  and  validate  the 
technical  aspects  of  deliverables  provided  by  the  systems  integrator 
and  other  vendors. 

The  AOTC's  failure  to  establish  strong  project  management  has  resulted  in 
lengthy  project  delays,  cost  overruns,  and  ad  hoc  project  planning.  After  operating 
without  an  IT  Plan  for  five  years  and  expending  $40  million,  the  AOTC  is  in  the  process 
of  selecting  a  new  project  manager  and  has  revised  its  IT  Plan.  The  Bureau,  however, 
doubts  that  the  AOTC  will  be  able  to  complete  the  project  within  the  time  specified  in  the 
Revised  IT  Plan  and  for  the  $75  million  allocated  by  the  Legislature.  In  support  of  this 
doubt,  the  Bureau  found: 

►  The  new  project  manager  does  not  have  a  detailed  plan  for  the 
daily  management  of  the  project; 

►  The  AOTC  is  seeking  a  new  vendor  that  will  have  to  verify  the 
substantial  work  performed  by  the  systems  integrator; 

►  The  AOTC  does  not  know  if  MassCourts  5  will  be  an  adaptation  of 
an  existing  commercially  available  system  ("Buy")  or  if  it  will  be 
specifically  developed  ("Build")  for  the  AOTC;  and 

►  The  history  of  the  project  and  the  budgetary  estimates  prepared  by 
a  project  consultant  casts  doubt  as  to  whether  the  project  will  be 
completed  as  specified  within  the  "System  Scope  document"  for 
$75  million. 

As  a  result  of  these  findings,  the  Bureau  makes  a  series  of  recommendations  that 
are  based  upon  accepted  management  practices  adopted  by  the  United  States  General 


4  When  referring  to  the  management  office  for  the  Trial  Court,  the  Bureau  will  use  the  acronym,  "AOTC." 
When  referring  to  the  Trial  Court  as  an  entity,  the  Bureau  will  use  the  words,  "Trial  Court." 

5  MassCourts  is  the  name  given  to  the  "comprehensive  integrated  information  system"  that  is  the  eventual 
product  of  this  project. 
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Accounting  Office's  Accounting  and  Information  Management  Division.6  The  Bureau 
recommends  that  the  AOTC  establish  a  strong  and  clear  management  structure  for  the  IT 
Project.  In  addition,  the  Bureau  recommends  that  the  AOTC  develop  comprehensive 
policies  and  procedures  to  guide  the  organization  through  the  remainder  of  the  project. 
Because  the  AOTC  presently  lacks  the  strong  management  structure  required  to  complete 
the  project  within  budget,  the  Bureau. recommends  that  the  AOTC  suspend  further 
development  of  the  "comprehensive  integrated  information  system"  until  such  time  as  it 
has  created  the  structure  and  developed  the  policies  that  are  needed  to  complete  the 
project  within  a  reasonable  time  and  for  the  money  budgeted. 


See,  Appendix  A,  infra. 


Objective,  Scope  and  Methodology 


As  directed  by  the  Massachusetts  House  of  Representatives  Post  Audit  and 
Oversight  Committee,  the  Massachusetts  House  of  Representatives  Post  Audit  and 
Oversight  Bureau  (Bureau)  issues  this  preliminary  report.  The  Bureau  seeks  to  determine 
whether  the  (AOTC)  complied  with  the  planning  and  reporting  provisions  of  c.  263  of  the 
Acts  of  1995,  as  amended.  The  Bureau  also  seeks  to  review  and  analyze  the  AOTC's 
management  of  the  IT  Project  in  light  of  its  findings  of  non-compliance  with  all  but  one 
of  the  planning  and  reporting  requirements. 

The  Bureau's  focus  is  on  the  AOTC,  and  the  inquiry  was  addressed  to  any  and  all 
management  level  personnel  within  the  AOTC  involved  with  the  IT  Project.  The  Bureau 
utilized  the  assistance  of  the  Information  Technology  Division  (ITD)  of  the  Executive 
Office  of  Administration  and  Finance  (A&F)  for  an  overview  of  Information  Technology 
(IT)  policies,  procurement  and  procedures.  In  addition,  the  Bureau  requested  any  and  all 
reports,  plans,  schedules  or  other  documentation  filed  by  the  AOTC  with  the  legislative 
committees  named  within  the  act,  A&F  and  with  ITD. 

-7 

The  Bureau  utilized  informal   evidence  gathering  techniques,  such  as: 

a.  Requesting  AOTC's  statement  concerning  compliance  with  the  provisions 
of  the  bond  bill; 

b.  Requesting  documents  filed  by  the  AOTC  with  legislative  committees  and 
with  A&F; 

c.  Requesting  documents  filed  with  ITD  pursuant  to  bond  acts  and 
interviewing  ITD  personnel  relative  to  its  interaction  with  the  AOTC 
concerning  the  expenditure  of  funds  from  the  act; 

d.  Requesting  documents  from  the  State  Comptroller  regarding  expenditures 
related  to  the  bond  bill; 

e.  Interviewing  AOTC  personnel  relative  to  the  IT  Project; 

f.  Interviewing  IT  Project  "systems  integrator"; 

g.  Reviewing  the  AOTC  IT  Project  files. 

In  addition,  the  Bureau  reviewed  and  analyzed  the  legislative  history  of  the  bond 
bill  and  subsequent  amendments  thereto.  The  Bureau  also  reviewed  IT  Management 
Practices  of  a  number  of  state  governments  and  the  federal  government. 


6.  Informal  evidence  gathering  techniques  are  those  techniques  which  are  utilized  without  resort  to 

the  Committee's  subpoena  power  as  provided  within  G.L.  c.  3,  §  63. 
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Background 

A.  The  Administrative  Office  of  the  Trial  Court  (AOTC) 

1  o 

The  AOTC  was  created  by  the  Legislature  in  accordance  with  its  constitutionally 
prescribed  "full  power  and  authority  to  erect  and  constitute  judicatories  and  courts  of 
record,  or  other  courts  to  be  held  in  the  name  of  the  Commonwealth. .  .."9.  The  AOTC's 
Chief  Justice  of  Administration  and  Management  ("CJAM")  is  likewise  a  judicial  office 
created  by  the  Legislature  the  responsibilities  of  which  -  as  "administrative  head  of  the 
Trial  Court"-  shall  be  "general  superintendence  of  the  administration  of  the  trial  court, 
including  without  limitation,  the  improvement  of  the  administration  of  such  courts  and 
the  securing  of  their  proper  and  efficient  administration."1     Specific  authority  and 
responsibility  as  it  relates  to  the  AOTC's  Information  Technology  (IT)  Plan,  which  is  the 
subject  of  the  present  inquiry,  includes  the  following: 

•  St.1992,  c.133,  §37; 

•  St.  1995,  c.263: 

•  The  responsibility  to  provide  the  departments  of  the  Trial  Court  with  technical 
assistance  concerning  record  keeping,  auditing  and  computers,  and  with  support 
services,  such  as  computerized  legal  research. . . 

•  The  authority  to  coordinate  the  development  and  maintenance  of,  and  technical 
assistance  for,  information  systems; 

•  The  responsibility  to  monitor  and  to  assist  in  the  case  processing  and  case  flow 
management  capabilities  of  the  Trial  Court  Departments; 

•  The  responsibility  to  provide  administrative  management  to  the  Office  of  the  Jury 
Commissioner; 

•  The  responsibility  to  review  and  make  recommendations  regarding  the 
expeditious  clearing  of  outstanding  warrants  throughout  the  courts  of  the 
Commonwealth. ]  l 

B.  Judiciary  Information  Technology  Loan,  Act  of  1 995 

Prior  to  1992,  the  individual  departments  of  the  Trial  Court  were  responsible  for 
the  design,  development,  procurement  and  maintenance  of  their  respective  information 
technology  (IT)  systems.  This  fragmentation  of  IT  systems  extended  to  individual 
divisions  within  a  department.  In  the  early  1990's  the  District  Court  Department 
developed  an  information  system  for  use  in  the  Brockton  and  Barnstable  District 


8  Constitution  of  Massachusetts,  Part  2,  Chap.  1,  Sec.  1,  Art.  3 
9G.L.c.  21  IB,  s.  1 


G.L.c.  211B,s.  9 
"G.L.C.  211B,s.  9 


Courts.     This  system,  however,  was  not  extended  nor  utilized  on  a  department-wide 
basis.  Similarly,  the  Superior  Court  Department  and  all  other  departments  within  the 
Trial  Court  implemented  their  own  information  systems  that  provided  applications  each 
of  which  were  different  than  and  incompatible  with  those  provided  by  the  other 
Departments. 

In  1992,  the  Legislature  froze  the  continued  expenditure  of  funds  pending  the 
creation  of  a  plan  for  an  integrated  computer  system  and  the  creation  of  a  single 
information  technology  office  for  the  AOTC.13  The  1992  legislation  also  directed  that 
any  future  spending  on  "computer  systems"  must  "conform  to  such  plan  and  shall  be 
subject  to  the  approval  by  the  chief  administrative  justice  to  ensure  such  conformity  with 
such  plan.    " 

Following  the  directives  of  the  1992  legislation,  the  AOTC  utilized,  in  1993,  two 
consulting  firms,  the  Point  Group  and  Knapp  +  Hoggshead,  to  assist  with  the  Information 
Technology  Plan  and  with  the  management  structure  imposed  by  statute.15  An  IT 
Planning  Study  was  conducted  that  focused  on  six  objectives: 

1 .  Define  the  IT  Vision; 

2.  Define  the  IT  Organization  and  Infrastructure; 

3.  Define  the  Strategic  Technical  Architectures; 

4.  Define  the  Current  Position  of  the  Trial  Court; 

5.  Define  a  Five  Year  IT  Action  Plan; 

6.  IT  Funding  Plan.16 

The  AOTC  solicited  a  request  for  information  from  technology  vendors  on 
the  scope  of  the  project  and  the  estimated  cost.17  The  proposals  indicated  that  the  project 
could  cost  between  $82  and  $98  million  to  complete.1    The  legislative  proposal  that  was 
forwarded  to  the  Legislature  by  the  AOTC  in  1995,  requested  $75  million  dollars  for  the 
project.19 

Chapter  263  of  the  Acts  of  1995  (bond  bill),  as  amended,  authorized  the 
expenditure  of  $75,000,000  by  the  AOTC  for  the  "acquisition,  upgrading,  development 
and  implementation  of  a  comprehensive  integrated  information  system  within  and 
between  the  departments  of  the  trial  court. .  ."20  The  bond  bill  also  established  an 
oversight  scheme  that,  if  followed,  would  allow  the  Legislative  and  the  Executive  branch, 
through  the  Secretary  of  A&F,  to  ensure  that  the  IT  Project  was  completed  within  budget 
and  on  schedule. 

C.        Information  Technology  Division 


12  Court  Administrator  Letter,  April  23,  1993. 
l3St.l992,c.l33,  §37. 

14  Id. 

15  SJC  Letter  to  Counsel  to  Governor,  September  1993. 

16  Point  Group  Master  Plan,  October  1993. 

17  RFI,  March  1994 

18  IT  Cost  Estimates,  June  1994. 

19  Id. 
20St.l995,c.  263. 


The  Information  Technology  Division  (ITD)  of  the  Executive  Office  of 
Administration  and  Finance  (A&F)  was  created  in  1 996  and  is  the  successor  to  the  Office 
of  Management  Information  Systems    (OMIS).  The  ITD  is  headed  by  the  chief 
information  officer  (CIO)  who  is  charged  with  setting  information  technology  standards, 
reviewing  and  approving  information  technology  strategic  plans,  reviewing  and 
approving  the  planning,  design,  acquisition  and  operation  of  information  technology 
systems  and  assessing  the  performance  of  information  technology  systems  and 
operations.22 

In  1998  the  Legislature  directed  that  ITD  "approve  any  schedule  of  expenditures 
proposed  by  any  agency  under  the  control  of  the  governor  for  any  software  development 
project  or  system  purchase  for  which  the  budgeted  cost  . . .  exceeds  $200,000."     In 
addition  to  this  direct  oversight  of  executive  branch  IT  system  projects,  ITD  exercises 
limited  control  over  IT  projects  for  non-executive  branch  projects  that  are  funded 
pursuant  to  IT  bonds.24    This  limited  control  is  exercised  via  the  bond  funds 
authorization  process. 

As  part  of  the  bond  funds  authorization  process,  the  Fiscal  Affairs  Division 
(FAD)  of  A&F  authorizes  annual  expenditures  from  the  various  IT  bond  bills  that  have 
been  enacted  by  the  Legislature.25  Upon  receipt  of  the  annual  allowance  from  FAD,  ITD 
determines  the  allocation  of  funds  among  the  various  IT  projects. 

In  order  to  make  funding  allocation  decisions,  ITD  has  instituted  an  "investment 
brief  process"  pursuant  to  which,  agencies  are  required  to  submit  investment  briefs. 
Because  the  bond  bill  authorizes  the  expenditure  of  funds  via  bonds,  AOTC's  IT  Project 
expenditures  must  be  processed  through  ITD.     As  part  of  ITD's  efforts  to  plan  and 
manage  the  Commonwealth's  information  technology  (IT)  investments,  agencies  are 
required  to  submit  investment  briefs  which  provide  a  project  overview,  the  strategic 
initiatives  that  are  supported  by  the  project,  the  project  need,  the  project  phase,  the 
business  plan  that  defines  the  objectives  that  the  project  is  expected  to  satisfy,  the  funding 
details  and  the  milestones  related  to  the  project. 


21  St.1996,  c.  151,  §  35,  and  codified,  G.L.  c.  7,  §  4A(d). 

22  G.L.  c.  7,  §  4A(d). 

23  St.  1 998,  c.  194 

24  As  such,  the  AOTC  is  required  to  submit  documentation  to  ITD  in  pursuit  of  expenditures  under  the  bond 
authorized  by  St.  1995,  c.  263. 

25  IT  Bond  Bill  I,  St.  1992,  c.  194,  §  2;  IT  Bond  Bill  II,  St.1996,  c.  294,  §  2;  DTA  Beacon,  St.  1992,  c.  194, 
§  2;  Education  Technology,  St.  1997.  c.  88,  §  2E;  Year  2000,  St.  1997.  c.  88,  §  2E  and  St.  1998,  c.  289; 
HR/CMS  Implementation,  St.1998,  c.  289;  Mini-Bond,  St.1995,  c.  96,  §  2;  Judiciary  Information 
Technology,  St.1995,  c.  263,  §  3. 

26  Bureau  Interview,  June  2001.  During  the  interview  with  ITD,  its  representatives  indicated  that  ITD  did 
not  have  the  same  approval  and  review  authorization  with  regard  to  the  trial  court  as  it  does  with  regard  to 
executive  branch  agencies  as  is  referenced  by  St.1998,  c.  194.  As  such,  ITD  has  relegated  its  review  of  the 
Trial  Court's  investment  briefs  to  an  analysis  of  the  total  amount  requested.  ITD  has  not  managed  or 
participated  in  the  planning  of  expenditures  for  the  Trial  Court  after  the  Trial  Court  is  authorized  to  expend 
funds  within  the  cap  established  by  ITD. 
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In  FY2002,  the  AOTC  submitted  its  first  substantially  complete  investment  brief 
which  was  prepared  by  the,  now  former,  Project  Manager.     However,  the  investment 
brief  submitted  for  FY2001  was  prepared  by  an  ITD  liaison  based  upon  information 
extracted  from  a  conversation,  or  conversations,  with  the  AOTC's  Chief  Information 
Officer  (CIO)  who,  by  his  own  admission,  "didn't  have  the  time"  to  complete  the 
investment  brief  although  he  was  aware  of  its  importance.  '  Neither  the  FY02  nor  FY01 
investment  brief  included  funding  details  or  project  milestones  for  the  underlying  project 
plan. 


27  ITD  first  implemented  the  Investment  Brief  process  for  use  in  FY2001. 

28  Bureau  Interview,  August  9,  2001. 


Chapter  1 ; 
Statutory  Compliance 

Summary  of  Findings 

i 

1 .  The  AOTC's  Failure  to  follow  the  IT  plan  violated  the  provisions  of 
St.1992,  c.133,  §37; 

2.  The  AOTC's  failure  to  file  schedules  of  expenditures  with  the 
legislature  violated  the  provisions  of  the  st.  1995,  c.  263; 

3 .  The  AOTC '  s  failure  to  file  yearly  accountings  and  assessments  with  the 
legislature  violated  the  provisions  of  the  st.1995,  c.  263; 

4.  The  AOTC's  failure  to  file  yearly  spending  plans  with  the  legislature 
violated  the  provisions  of  the  st.1995,  c.  263; 

5 .  The  AOTC '  s  failure  to  submit  an  adequate  accounting  of  costs  related 

TO  THE  ADMINISTRATION  OF  THE  PROJECT  VIOLATED  THE  PROVISIONS  OF  ST.  1995, 

C.  263; 

6.  The  AOTC  Failure  to  Inform  the  Legislature  that  It  was  Deviating  from 
the  1995  IT  Plan  as  a  result  of  reduced  bond  fund  allocations  violated 
St.1995,c.  263. 

Introduction 


"After  the  completion  of  such  study  and  plan,  all 
expenditures  . . .  shall  conform  to  such  plan  and  shall  be 
subject  to  the  approval  by  the  Chief  Administrative 
Justice  to  ensure  such  conformity  with  such  plan." 

-St.1992,  c.133,  §37. 


St.  1992,  c.  133  ("1992  legislation")  and  St.  1995,  c.  263  ("bond  bill")  imposed 
exacting  requirements  upon  the  Administrative  Office  of  the  Trial  Court  (AOTC).  These 
acts  required  that  information  technology  (IT)  plans  be  prepared  and  that  the  AOTC  act 
in  accordance  with  those  plans.  The  bond  bill  imposed  a  rigid  and  comprehensive 
reporting  regimen  upon  the  AOTC.  The  AOTC  was  required  to  inform  the  Legislature  of 
its  plans;  inform  the  Legislature  of  its  planned  expenditures;  inform  the  Legislature  of  the 
status  of  the  project;  and  to  what  extent,  if  any,  the  actions  of  the  AOTC  deviated  from 
the  plans. 


29 


See,St.l995,c.263. 
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Sections  two  and  five  of  the  bond  bill  established  procedures  that  would  allow  the 
Legislative  and  the  Executive  branches  of  government  to  oversee  the  project  on  an 
ongoing  basis.30  Section  two  of  the  bond  bill  also  required  that  an  expenditure  plan  be 
"filed  with  the  house  and  senate  committees  on  ways  and  means,  the  house  and  senate 
committees  on  science  and  technology,  the  joint  committee  on  state  administration  and 
the  secretary  of  administration  and  finance. . .  ("required  entities")."31  The  bond  bill  also 
provides  that  "no  funds  shall  be  expended  from  this  item  until  such  time  as  the  chief 
justice  for  administration  and  management  of  the  trial  court  submits  a  plan. . ." 
Following  completion  and  submission  of  the  above  plan,  any  expenditure  of  funds  would 
be  based  upon  "schedules  prepared  by  the  office  of  the  chief  justice. . ."  and  submitted  to 
the  required  entities.  "  The  submission  of  schedules  by  the  AOTC  is  required  to  be  made 
"thirty  days  prior  to  the  obligation  of  said  funds."34 

Beginning  not  later  than  September  1,  1996,  and  annually  thereafter,  the  AOTC 
was  required  to  file  with  the  required  entities,  an  "accounting  of  expenditures  ...  for  the 
prior  fiscal  year"  including  an  "accounting  of  the  amount  expended  for  hardware, 
software  and  personnel,  and  assessment  of  whether  such  project  is  within  budget  and  on 
schedule  for  completion,  and  an  explanation  of  any  deviations  in  completion  timetables 
and  funding  needs  that  were  originally  established  for  such  project."3    Similarly,  the 
AOTC  was  required  to  "simultaneously  file  ...  a  spending  plan  identifying  projected 
expenditures  on  such  project  in  the  current  fiscal  year  . . .  including  expenditures  by  type, 
and  projected  milestones  and  completion  schedules."36  Lastly,  the  Legislature  required 
that  "not  more  than  five  percent  of  the  funds  . . .  may  be  expended  for  the  administration 
and  implementation"  of  the  project.      In  order  to  monitor  this  requirement,  the  bond  bill 
required  the  AOTC  to  submit  a  "report  detailing  said  administrative  expenses  to  said 
committees."38 

This  legislative  oversight  scheme  and  its  emphasis  on  a  "plan"'"yas  consistent  with 
the  1992  legislation  that  mandated  that  any  future  spending  on  information  technology  be 
conducted  in  conformity  with  a  comprehensive  plan  and  that  the  CJAM  ensure  that 
expenditures  were  made  pursuant  to  the  comprehensive  plan.     However,  the  AOTC 
failed  to  comply  with  all  but  one  of  the  pertinent  adherence  and  reporting  requirements  of 
the  above  statutes.  This  complete  failure  evidences  and  illustrates  a  management 
problem  that  existed  and  continues  to  exist  within  the  AOTC  and  its  departments. 

As  an  example,  the  Bureau  was  informed  that  the  legal  department  of  the  AOTC 
regularly  produces  a  summary  of  each  legislative  enactment  that  affects  the  Trial  Court.40 
Each  AOTC  official  specifically  queried  about  her  or  his  knowledge  of  the  bond  bill's 
reporting  requirements  stated  that  they  were  unaware  of  the  requirements  and  that  they 


30 


St.  1995,  c.  263,  §§  2  and  5. 


31  Id. 

32  Id. 

33 


St.1995,  c.  263,  §5. 

34  Id. 

35  Id. 

36  Id. 

37  St.1995,  c.  263,  §2. 

38  Id. 

39St.l992,  c.133,  §37. 

40  Bureau  Interview,  September  12,  2001. 
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had  not  received  a  summary  or  analysis  from  the  AOTC  legal  department.41 
Notwithstanding  these  statements,  the  Bureau  discovered  two  separate  documents,  both 
prepared  in  1996  by,  that  provided  the  AOTC  and,  in  particular  its  CIO  who  was  then 
charged  with  IT  Project  management  responsibility,  with  a  comprehensive  analysis  of  the 
reporting  requirements  imposed  by  the  bond  bill.42 

The  AOTC  has  failed  to  provide  a  valid  explanation  as  to  why  the  filings  were  not 
made  and  why  the  Legislature  was  never  apprised  that  the  IT  plan  was  not  being 
followed.  Based  upon  the  documents  that  were  reviewed,  it  is  apparent  that  no  efforts 
were  made  to  establish  a  reporting  system  or  to  insure  that  the  project  proceeded 
according  to  the  plan  that  was  filed  with  the  Legislature.43 

The  failure  to  report  pursuant  to  the  bond  bill  is  more  troublesome  given  that  the 
AOTC  was  aware  at  the  outset  of  the  project  that  the  bond  bill  imposed  filing  and 
reporting  requirements.  In  fact,  the  AOTC  did  file  an  IT  plan  in  1995  and  initially  filed 
"schedules,"  so-called,  pursuant  to  the  reporting  requirements  found  within  the  bond  bill. 
However,  as  will  be  detailed  below,  compliance  and  disclosure  quickly  came  to  an  end. 

Findings: 

1 .    The  AOTC's  Failure  to  follow  the  IT  plan  was  in  violation  of  St.  1 992, 
C.133,  §37. 

Chapter  133  of  the  Acts  of  1992  provided  the  following: 

"The  Chief  Administrative  Justice  ...  shall  conduct  a  study  and  develop  a  plan  for 
the  most  effective  means  of  creating  an  integrated  information  system  for  the 
entire  trial  court,  including,  if  possible,  the  creation  of  a  single  management 
information  office  for  the  entire  trial  court...  After  the  completion  of  such  study 
and  plan,  all  expenditures  ...  shall  conform  to  such  plan  and  shall  be  subject  to 
the  approval  by  the  Chief  Administrative  Justice  to  ensure  such  conformity  with 
such  plan  ('emphasis  added). ' 

On  or  about  December  11,  1995  the  AOTC  filed  a  document  with  the  Legislature 
entitled  the  "Information  Technology  (IT)  Plan"  (1995  IT  Plan).  The  document 
contained  an  IT  Vision  statement;  a  breakdown  of  funding  requirements;  and  the 
Integrated  Information  System  Plan.  However,  the  AOTC  deviated  from  the  plan  and  did 
not  file  any  update  to  the  plan  until  September  5,  200 1.45 


41  Bureau  Interviews,  August  2,  2001,  August  9,  2001,  and  September  12,  2001. 

42  AOTC  Legal  memorandum  relative  to  St.  1995,  c.  263,  May  23,  1996;  and  ITD  Reporting  Requirements 
Memorandum,  September  of  1996. 

43  Assistant  AOTC  IT  Project  Manager  Notes,  July  16,  2001;  Bureau  Interviews,  August  2,  2001,  August  9, 
2001,  and  September  12,  2001. 

44  St.  1992,  c.133,  §37. 

45  See,  September  5,  2001  Legislative  Filing. 
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The  1995  IT  Plan  was  divided  into  five  tracks: 

1 .  Desktop  Implementation; 

2.  Network  Infrastructure; 

3.  Core  Application  Development; 

4.  Systems  Migration  and  Customization; 

5.  Disaster  Recovery. 

The  AOTC  has  deviated  from  each  of  the  above  tracks  and  failed  to  notify  the  Legislature 
of  the  deviation  until  September  5,  2001,  in  violation  of  the  provisions  of  the  1992 
legislation  and  the  bond  bill.  A  brief  analysis  of  the  project  tracks  reveals  the  following: 

Desktop  Implementation 

Upon  the  receipt  of  bond  funds  in  Fiscal  Year  1997,46  the  AOTC  began  an 
expansive  program  of  desktop  and  laptop  computer  purchases  for  the  Trial  Court.  These 
purchases,  however,  were  not  made  pursuant  to  the  1995  IT  Plan  and  there  was  no 
apparent  analysis  concerning  the  potential  obsolescence  of  the  purchased  computers  by 
the  time  the  core  application  was  developed.  Lastly,  no  analysis  was  prepared 
concerning  the  core  application's  potential  impact  upon  the  technical  specifications  and 
requirements  of  the  desktops.47 

Network  Infrastructure 

The  AOTC  failed  to  comply  with  the  network  infrastructure  track  of  the  plan.  As 
of  the  date  of  this  report,  the  Bureau  has  been  presented  with  conflicting  statements 
concerning  the  existence  and  extent  of  a  comprehensive  network  topography  plan.     The 
AOTC  has  not  prepared  a  comprehensive  analysis  of  the  readiness  of  the  court  facilities 
throughout  the  Commonwealth  for  the  interim  systems  or  for  the  core  application.49 
Finally,  the  AOTC  has  deferred  establishing  a  network  procurement  and  vendor  selection 
plan. 

Core  Application  Development 

The  AOTC  failed  to  comply  with  the  core  application  development  track  of  the 
plan.  As  of  the  date  of  the  report,  the  AOTC  has  yet  to  issue  a  request  for  responses  for 
the  development  of  the  core  application  now  known  as  MassCourts.  Recently  published 
reports,  quoting  the  Project  Director,  state  that  "the  RFR  will  likely  be  issued  later  this 
year"  [2001],  but  "no  specific  timetable  is  currently  in  place  for  the  completion  of  the 
RFR  process.50" 


46  Comptroller's  Office  #0330-0951. 

47  Bureau  Interview,  August  9,  2001. 

48  Bureau  Interview,  August  2,  2001,  October  4,  2001,  November  5,  2001 . 

49  Bureau  Interview,  October  4,  2001,  November  5,  2001. 

50  Massachusetts  Lawyers  Weekly,  November  5,  2001. 
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Systems  Migration  and  Customization 

The  AOTC  failed  to  comply  with  the  systems  migration  and  customization  track 
of  the  plan  although  the  AOTC  did  expand  and,  in  some  cases,  enhance  the  existing 
systems  within  the  departments  of  the  Trial  Court,  primarily  in  response  to  Y2Krelated 
concerns.  However,  this  program  for  the  expansion  and  enhancement  of  the  existing 
systems  has  not  proceeded  according  to  the  schedule  established  within  the  1995  IT  Plan. 

Disaster  Recovery 

The  AOTC  failed  to  comply  with  the  disaster  recovery  track  of  the  plan.  As  of 
the  date  of  this  report,  the  AOTC  has  yet  to  develop  a  disaster  recovery  plan  and 
system,51  deciding  instead  to  wait  until  development  of  the  core  application  -  MassCourts 
-  is  completed. 

As  such,  the  AOTC  failed  to  adhere  to  the  IT  Plan  filed  with  the  Legislature. 

Highlighting  the  AOTC's  failure  to  follow  the  IT  Plan  is  the  fact  that,  from  1995 
through  the  initiation  of  the  inquiry  by  the  Bureau,  the  AOTC  failed  to  reference  the  plan 
or  to  incorporate  the  document  into  its  project  related  actions.  In  fact,  there  appears  to  be 
a  widespread  understanding  that  no  viable  IT  Plan  existed.  For  example, 

►  In  December  of  1 996,  the  co-chairs  of  the  high  level  Information 
Technology  Advisory  Committee  (IT AC)  stated  that  a  plan  would  be 
necessary  so  that  project  progress  could  be  gauged  and  measured.  These 
statements  were  made  one  year  after  the  IT  Plan  had  been  filed  with  the 
Legislature.53 

►  In  March  of  2000,  the  Judge  who  had  been  appointed  the  Project  Director 
informed  the  Wall  Street  Journal  that  there  "was  no  formal  plan,  but  rather 
a  set  of  evolving  'working  documents."'54 

►  In  August  of  2001 ,  the  former  Project  Manager  informed  the  Bureau  that 
she  was  not  aware  of  a  formal  IT  Plan.  The  Bureau  presented  the 
December  1995  document  to  the  individual  who  stated  that  was  the  first 
time  she  had  seen  the  plan.55 

►  In  October  of  2001,  the  Chair  of  the  Project  Advisory  Board  (PAB),  the 
internal  entity  charged  with  oversight  for  the  project,  stated  that  the  PAB 
had  not  reviewed  an  IT  Plan  and  that  the  AOTC  was  not  following  an  IT 
Plan.56 


51  Bureau  Interview,  August  9,  2001 . 

52  See,  September  5,  2001  Legislative  Filing. 

53  IT  AC  Co-Chair  Meeting  Memorandum,  December  21,  1996. 

54  Wall  Street  Journal,  Judges  and  Lawyers  Object  to  Tech  Team  's  Methods,  Wednesday,  March  22,  2000. 

55  Bureau  Interview,  August  2,  2001. 

56  Bureau  Interview,  October  4,  2001 . 
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While,  the  1995  IT  Plan  that  was  filed  with  the  Legislature  technically  satisfied 
the  requirements  of  the  bond  bill,  insofar  as  it  was  filed,  the  AOTC  thereafter  engaged  in 
an  apparently  unstructured  pattern  of  IT  development  that  was  unrelated  to  any  IT  Plan.57 
The  failure  by  the  AOTC  to  adhere  to  the  plan  or  to  amend  the  plan  violated  the  explicit 
provisions  of  the  bond  bill  and  the  1992  legislation. 

2.    The  AOTC's  failure  to  file  schedules  of  expenditures  with  the 
legislature  violated  the  provisions  of  the  st.1995,  c.  263 

The  Judicial  Information  Technology  Loan  Act  of  1995  provided  the  following: 

"expenditure  of  funds  ...shall  be  based  upon  "schedules  prepared  by  the 
office  of  the  chief  justice...  and  submitted  ...thirty  days  prior  to  the 
obligation  of  said  funds.  "58 

The  AOTC  has  failed  to  provide  the  Legislature  with  the  schedules  required  by 
the  bond  bill. 

The  bond  bill  specifically  required  that  the  AOTC  assume  the  responsibility  for 
the  submission  of  schedules  of  expenditures  to  the  required  entities.59  The  imposition  of 
this  filing  requirement  upon  the  AOTC  was  included  at  its  insistence.60  Earlier  drafts  of 
the  legislation  had  directed  the  Secretary  of  A&F  to  submit  all  required  plans,  schedules 
and  other  documentation  to  the  legislative  committees  relative  to  the  AOTC's  bond  bill.61 
The  AOTC  resisted  this  arrangement,  arguing  that  the  AOTC  -  a  component  of  the 
Judicial  Branch  -  should  not  be  subject  to  the  direct  oversight  from  the  Executive 
Branch.     As  such,  the  language  was  altered  to  require  filing  from  the  AOTC  and  only 
minimal  consultation  with  OMIS,  a  division  of  A&F.    Notwithstanding  the  express 
requirements  of  this  provision,  the  AOTC  last  submitted  a  schedule  to  the  Legislature  in 
December  of  1997.  Since  December  of  1997,  the  schedules  have  been  submitted  by  ITD. 

The  "schedules"  that  have  been  submitted  pursuant  to  the  bond  bill  by  ITD  have  been 
declarations  of  the  total  yearly  amount  of  bond  funds  that  were  allocated  to  the  AOTC  by 
ITD.64  By  way  of  comparison,  schedules  submitted  by  ITD  relative  to  other  Information 


57  Bureau  Interview,  August  9,  2001 .  In  defending  this  approach,  the  AOTC's  CIO  stated  that  he  and  the 
AOTC  had  a  "consciousness"  of  where  the  AOTC  was  going  with  the  IT  project. 
58St.l995,c.263,  §5. 

59  Id. 

60  See,  SJC  Administrator's  Comments  Regarding  Bond  Bill  Language,  June  of  1994;  and,  the  Point  Group 
Bond  Bill  Language  Memorandum,  June  of  1994. 

61  House  #5427,  (1995);  An  AOTC  legal  memorandum,  a  consultant  memorandum  and  interviews  with 
AOTC  officials  support  the  view  that  the  AOTC  perceived  the  active  involvement  of  the  executive  branch 
of  government  in  this  project  as  an  infringement  upon  the  separation  of  powers  between  the  branches.  See, 
Point  Group  Bond  Bill  Language  Memorandum,  June  of  1994;  AOTC  Legal  Memorandum  concerning 
St.1995,  c.263,  May  23,  1996;  and,  Bureau  Interviews,  August  2,  2001,  August  9,  2001,  and  September  12, 
2001. 

62  See,  SJC  Administrator's  Comments  Regarding  Bond  Bill  Language,  June  of  1994;  and,  the  Point  Group 
Bond  Bill  Language  Memorandum,  June  of  1994. 

63  Compare  House  #5427  (1995)  and  St.1995,  c.263,  §5. 

64  Bureau  Interview,  June  27,  2001 .  During  interviews  with  representatives  from  ITD,  those  representatives 
stated  that  they  have  not  been  provided  with  a  detailed  schedule  of  expenditures  from  the  AOTC.  The 
representatives  indicated  that  ITD  is  cautious  in  its  dealings  with  the  AOTC  because  it  does  not  want  to 

15 


Technology  bond  bills  provide  a  greater  level  of  specificity  concerning  how  funds  are  to 


65 


be  expended.  "'  ITD  could  not  provide  greater  specificity,  relative  to  the  AOTC,  because 
the  AOTC  failed  to  provide  any  detailed  information  to  ITD  concerning  project  plans  or 
expenditures 
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The  bond  bill  does  not  specifically  define  what  level  of  information  is  required  within 
the  "schedules."'     However,  given  the  planning  and  oversight  scheme  established  by  the 
bond  bill,  it  is  reasonable  to  require  the  AOTC  to  provide  a  level  of  specificity  that  would 
allow  the  required  entities  to  measure  expenditures  against  filed  spending  plans.68  A 
greater  level  of  detail  is  supported  by  the  plain  meaning  of  the  word  "schedule."  The 
commonly  accepted  definition69  of  the  word  requires  more  than  a  mere  recitation  of  the 
annual  amount  of  funds  to  be  expended.  For  purposes  of  the  schedules  filing 
requirement,  the  Bureau  recommends  that  a  schedule  include  a  detailed  itemization  of  the 
bond  funds  that  will  be  expended  and  what  will  be  purchased  with  the  said  funds 
pursuant  to  the  IT  Plan.  This  form  of  filing  would  comport  with  the  common  usage  of 
the  word  "schedule"  and  with  the  legislative  oversight  scheme  established  within  the 
bond  bill.70 

Therefore,  the  AOTC's  failure,  since  1997,  to  file  schedules  with  the  required  entities 
violated  the  provisions  of  St. 1995,  c.263,  §5.  Similarly,  as  a  result  of  the  AOTC's  failure 
to  provide  sufficient  information,  the  schedules  that  have  been  filed  by  ITD  have  not 
contained  the  detail  implicitly  required  by  the  bond  bill. 

3.   The  AOTC's  failure  to  file  yearly  accountings  and  assessments  with  the 
legislature  violated  the  provisions  of  the  st.1995,  c.  263 

The  Judicial  Information  Technology  Loan  Act  of  1995  provided  the  following: 

"Accounting  of  expenditures  for  said  project ...  included  but  not  limited  to,  an 
accounting  of  the  amount  expended  for  hardware,  software  and  personnel,  an 
assessment  of  whether  such  project  is  within  budget  and  on  schedule  for 
completion,  and  an  explanation  of  any  deviation  in  completion  timetables  and 
funding  needs  that  were  originally  established  for  such  project.  " 

The  AOTC  failed  to  file  the  required  yearly  accounting  and  status  assessments 
from  the  inception  of  the  project's  funding  in  FY  1997  through  and  including  FY  2001. 
The  first,  and  only,  accounting  and  assessment  was  filed  with  the  Legislature  in 
September  of  2001,  after  the  initiation  of  the  present  inquiry  by  the  Committee  and  its 
Bureau.73 


overstep  constitutional  parameters.  As  such,  ITD  has  not  requested  detailed  spending  plans  or  project 
plans. 

65  Bureau  Interview,  June  27,  2001. 

66  Bureau  Interview,  June  27,  2001. 
67St.l995,  c.263,  §5. 

68  See,  Id. 

69  Webster's  Dictionary 
70St.l995,c.263,§5.  " 

71  Id. 

72  Id. 

73  See,  September  5,  2001  legislative  filing. 
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All  AOTC  project  officials  interviewed  by  the  Bureau  stated  that  they  were 
unaware  of  the  filing  requirements  imposed  by  the  Legislature  via  the  bond  bill.  Most 
officials  interviewed  stated  that  they  first  became  aware  of  this  requirement  when  so 
informed  by  the  Bureau.      These  assertions  are  troubling  given  the  following  facts: 

►  The  legal  department  of  the  AOTC  prepared  a  comprehensive  analysis  of 
the  bond  bill  for  the  AOTC's  CIO  in  May  of  1996.  The  analysis  provided 

"7  *\ 

the  full  scope  of  all  reporting  requirements  under  the  statute.  " 

►  The  CIO  of  the  AOTC  was  also  briefed  in  September  of  1 996  by  the  Chief 
Information  Officer  of  ITD  concerning  the  reporting  requirements 
provided  in  the  legislation.7 

As  such,  the  CIO  of  the  AOTC  and  the  Legal  Department  for  the  AOTC  were 
aware  of  the  requirements  imposed  by  the  Legislature,  but  failed  to  comply  for  a  period 
of  five  years.  In  addition,  the  AOTC's  December  1995  filing  of  the  IT  Plan  and  the 
submission  of  schedules  to  the  required  entities  in  1997,  demonstrate  that  the  AOTC  was 
aware  that  the  bond  bill  imposed  stringent  filing  and  reporting  requirements.  No  valid 
explanation  has  been  provided  to  the  Bureau  concerning  why  any  compliance  with  the 
planning  and  reporting  provisions  of  the  bond  bill  ended  in  1997. 

Therefore,  the  AOTC's  failure  to  file  yearly  accountings  and  status  assessments 
violated  the  provisions  of  St. 1995,  c.263,  §5. 
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4.    The  AOTC's  failure  to  file  yearly  spending  plans  with  the  legislature 
violated  the  provisions  of  the  st.1995,  c.  263 

The  Judicial  Information  Technology  Loan  Act  of  1995  provided  the  following: 

"A  spending  plan  identifying  projected  expenditures  on  such  project  in  the 
current  fiscal  year,  including  but  not  limited  to  expenditures  by  type,  and 
projected  milestones  and  completion  schedules.  " 

The  AOTC  failed  to  file  the  yearly  spending  plans  that  were  required.  The  first, 
and  only,  spending  plan  was  filed  with  the  Legislature  in  September  of  2001  after  the 
initiation  of  the  present  inquiry  by  the  Committee  and  its  Bureau.7    As  stated  above,  the 
AOTC  had  been  briefed  in  May  and  September  of  1996,  yet  failed  to  comply  with  the 
requirements  of  the  statute  for  a  period  of  six  years. 


74  Bureau  Interviews,  August  2,  2001,  August  9,  2001,  September  12,  2001,  September  28,  2001,  and 
October  4,  2001. 

75  AOTC  Legal  Memorandum  Relative  to  St.1995,  c.  263,  May  23,  1996. 

76  ITD  Reporting  Requirements  Memorandum,  September  of  1996. 

77  St.1995,  c.263,  §5. 

78  Id. 

79  See,  September  5,  2001  Legislative  Filing. 
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Therefore,  the  AOTC's  failure  to  file  yearly  spending  plans  violated  the 
provisions  of  St.  1995,  c.263,  §5.80 

5 .   The  AOTC '  s  failed  to  submit  an  adequate  accounting  of  costs  related 

TO  THE  ADMINISTRATION  OF  THE  PROJECT  VIOLATED  THE  PROVISIONS  OF  ST.  1 995, 

c.  263 

The  Judicial  Information  Technology  Loan  Act  of  1995  provided  the  following: 
"No  more  than  five  percent  of  the  funds  authorized  herein  may  be  expended  for 
the  administration  and  implementation  of  said  capital  projects  funded  herein  ... 
said  chief  justice  shall  submit  a  report  detailing  said  administrative  expenses... 


,81 


The  AOTC  failed  to  submit  a  report  detailing  the  administrative  expenses  for  the 
project.  The  first,  and  only,  such  report  was  submitted  to  the  Legislature  in  September  of 
2001  after  the  initiation  of  the  present  inquiry  by  the  Committee  and  its  Bureau. 

The  bond  bill  does  not  specifically  define  what  are  costs  related  to  administration 
and  implementation.      In  addition,  the  AOTC,  prior  to  July  of  2001,  did  not  define  what 
project  expenses  would  be  categorized  as  being  related  to  administration  and 
implementation.84  The  AOTC's  September  5,  2001  legislative  filing,  however,  does 
provide  a  list  of  project  expenses  that  are  categorized  as  "administrative.    "  In  reviewing 
the  list  of  administrative  expenses  provided,  the  Bureau  finds  that  the  report  does  not 
sufficiently  cover  the  entire  scope  of  administrative  expenses  nor  does  it  itemize  the 
expenses  in  a  particularized  fashion  or  provide  costs  related  to  implementation. 

Because  the  provision  is  not  defined  and  the  AOTC  has  not  established  an  internal 
mechanism  to  specifically  define  what  are  costs  related  to  administration  and 
implementation,    the  traditional  rules  of  statutory  construction  provide  that  words  should 
be  given  their  plain  and  ordinary  meaning  unless  doing  so  would  do  violence  to  the 
obvious  intention  of  the  Legislature. 

Webster's  and  Black's  Dictionaries  provide  consistent  definitions  that  would 
support  the  intent  of  the  legislative  enactment  as  a  whole. 

1 .    Administration:  [Webster's]  (1 )  the  act  or  process  of  administering  or  (2) 
performance  of  executive  duties;  administer[ing]:  to  manage  or  supervise  the 
execution,  use  or  conduct  of  [a  trust  fund];  [Black's]  administrative:  . . . 
pertains  to  administration,  especially  management,  as  by  managing  or 
conducting,  directing,  or  superintending,  the  execution,  application  or  conduct 
of  persons  or  things. 


80  St.  1995,  c.263,  §5. 

81  St.1995,  c.263,  §2. 

82  See,  September  5,  2001  Legislative  Filing. 

83  St.1995,  c.263,  §2. 

84  Assistant  AOTC  IT  Project  Manager  Notes,  July  16,  2001 . 

85  See,  September  5,  2001  Legislative  Filing. 

86  Bureau  Interviews,  August  9,  2001,  and  September  12,  2001 . 
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2.    Implementation:  [Webster's]  to  carry  out  or  accomplish;  give  practical  effect 
to  and  ensure  actual  fulfillment  by  concrete  measures. 

In  view  of  these  definitions,  the  Bureau  recommends  that  costs  related  to 
administration  and  implementation  should  include  the  full  cost  of  the  AOTC's  Project 
Office,  the  function  and  purpose  of  which  should  be  limited  to  the  management, 
supervision  and  actual  fulfillment  of  the  bond  bill's  stated  purpose.  In  addition,  costs 
related  to  the  administration  of  the  oversight  committees  and  sub-committees  as  well  as 
any  consultant  contract  costs  associated  with  the  discharge  of  the  functions  of  the 
oversight  committees  and  sub-committees  should  be  the  benchmark  by  which  compliance 
with  the  $3.5  million  cap  on  administration  and  implementation  expenses  should  be 
measured. 

As  mentioned  earlier,  the  AOTC  has  submitted  only  one  report  on  the 
administration  costs  of  the  project  notwithstanding  the  fact  that  the  AOTC  has  been 
receiving  and  expending  bond  funds  since  FY  1997.  Also,  the  report  does  not  sufficiently 
cover  the  entire  scope  of  expenses  nor  does  it  itemize  the  expenses  in  a  particularized 
fashion.88 

Therefore,  the  AOTC's  failure  to  submit  a  report  of  administration  and 
implementation  costs  violated  the  provisions  of  St.  1995,  c.263,  §2.  Similarly,  the  report 
that  was  submitted  in  September  of  2001  does  not  include  the  entire  scope  of  expenses 
that  should  be  apportioned  to  administration  and  implementation. 

6.   The  AOTC  Failed  to  Inform  the  Legislature  that  It  was  Deviating  from 
the  1995  IT  Plan  as  a  result  of  diminished  bond  fund  allocations. 

The  AOTC  did  not  inform  the  Legislature  that  the  IT  Project  was  being  delayed  as  a 
result  of  bond  funds  being  allocated  in  lesser  amounts  than  were  anticipated.  Similarly, 
the  AOTC  did  not  revise  its  IT  Plan  after  realizing  that  bond  funds  would  be  distributed 
in  amounts  equal  to  approximately  50%  of  anticipated  annual  allocations. 

In  the  Revised  IT  Plan  filed  with  the  Legislature  on  September  5,  2001,  the 
AOTC  states  that  it  deviated  from  the  1995  IT  Plan  because  bond  funds  were  not  released 
as  originally  contemplated  and  as  set  forth  in  the  plan.90  Because  ITD  was  allocating 
funds  at  a  lower  rate  than  expected,  "certain  expectations  and  assumptions  as  contained  in 
the  plan  had  to  change."91  The  explanation  of  deviations  lists  a  number  of  project 
decisions  that  were  made  to  deal  with  the  new  funding  reality.  These  include: 

►  changing  the  development  approach  from  "big  bang"  to 
"iterative"; 

►  hiring  a  consultant  to  prepare  a  detailed  infrastructure 
design; 


87  By  June  of  2001,  forty  percent  of  the  administrative  costs  relate  to  salaries. 

88  See,  September  5,  2001  Legislative  Filing. 

89  St.1995,  c.263,  §2.  AOTC  Legal  Memorandum,  May  23,  1996. 

90  See,  September  5,  2001  legislative  filing  at  section  3. 

91  Id.  at  Section  3,  page  3. 
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►  hiring  a  systems  integrator; 

►  distributing  notebook  computers  to  judges; 

►  outfitting  new  courts  with  computer  technology;  and 

►  development  of  interim  systems. 

The  AOTC  states  that  these  programmatic  decisions  were  "consistent  with  the  overall 
plan  of  action,  these  funds  were  used  to  upgrade  central  computing  hardware  and 
software."      While  the  assertion  may  be  accurate,  it  is  not  supported  by  any  written  IT 
Plan. 

As  mentioned  above,  the  AOTC  was  aware  in  May  of  1 996,  prior  to  the 
enactment  of  the  "terms  bill"93  that  authorized  the  distribution  of  funds,  that  the 
Legislature  mandated  that  an  IT  Plan  be  filed  and  that  spending  on  the  project  comport 
with  the  filed  plan.94  In  addition,  the  1992  legislation  mandated  that  any  IT  related 
spending  "shall  conform  to  such  plan  and  shall  be  subject  to  the  approval  by  the  Chief 
Administrative  Justice  to  ensure  such  conformity  with  such  plan."95  The  AOTC  legal 
department  advised  the  AOTC  that  should  the  AOTC  encounter  any  "executive  obstacles 
to  the  desirable  implementation  of  St.1995,  c.263  and  be  unable  to  achieve  their 
negotiated  resolution,  then  the  judicial  branch's  recourse  is  to  the  Legislature  for 
appropriate  statutory  changes."96 

Here,  when  faced  with  a  reduced  bond  fund  allocation,  the  AOTC  did  not  inform 
the  Legislature  of  the  anticipated  impact  and  did  not  prepare  a  revised  plan  to  address  the 
fiscal  realities  imposed  by  the  bond  cap.  Conceding  that  the  reduced  yearly 
appropriations  impacted  the  timelines  initially  filed  with  the  Legislature  in  1995,  the 
AOTC  had  an  obligation  to  immediately  prepare  a  revised  plan.     This  obligation  was 
based  upon  the  1992  legislation  and  the  bond  bill.98  The  AOTC  did  not  prepare,  nor  file, 
a  Revised  Plan  until  September  5,  2001."  Therefore,  more  than  $40  million  has  been 
expended  without  any  apparent  plan.  While  the  assertion  that  project-spending  decisions 
were  "consistent  with  the  overall  plan  of  action. .  ."10°  may  lessen  concern,  the  reality  is 
that  public  funds  were  expended  without  a  clear,  defined,  spending  plan  as  was  required 
by  statute.  While  all  of  the  expenditures  and  decisions  that  were  made,  may  be  of  utility 
to  the  final  comprehensive  integrated  system,  these  expenditures  were  made  contrary  to 
the  articulated  mandates  of  the  Legislature.101 

Recommendations : 


92  Id.  at  Section  3,  page  9. 
,3St.l996,c.l86,  §3. 


94 


St.l992,c.l33,  §37. 


95  Id. 

96  AOTC  Legal  Memorandum  Relative  to  St.1995,  c.  263,  May  23,  1996. 
97St.l992,c.l33,§37. 

98  St.1995,  c.263;  and,  St.  1992,  c.133,  §37. 

99  See,  September  5,  2001  Legislative  Filing. 

100  Id.  at  Section  3,  page  9. 

101  St.1995,  c.263;  and,  St.1992,  c.133,  §37. 
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To  improve  the  AOTC's  ability  to  comply  with  statutory  directives,  in  general,  and  as 
they  relate  to  St.  1992,  c.133,  §37  and  St.  1995,  c.263,  in  particular,  the  AOTC  should  take 
the  following  actions: 

Develop  a  written  policy  that  would  guarantee  that  legislative  analysis  and  a 
statement  of  statutory  directives  would  be  disseminated  to  all  relevant  and 
appropriate  individuals  within  the  Trial  Court; 

Develop  a  written  policy  that  would  ensure  compliance  with  statutory  directives 
and  would  provide  an  internal  oversight  mechanism  to  ensure  compliance; 

Develop  an  appropriate  schedule  of  anticipated  expenditures  that  would  provide 
the  level  of  detail  required  by  the  provisions  of  St.  1995,  c.  263; 

•  Develop  a  process  that  would  allow  the  CJAM  to  verify  that  any  and  all  IT 
expenditures  were  made  pursuant  to  an  IT  Plan,  as  is  required  by  the  provisions  of 
St.1992,  c.133,  §37  and  St.1995,  c.263; 

•  Develop  a  process  that  would  ensure  that  the  AOTC  file  yearly  accountings, 
assessments  and  spending  plans  with  the  required  entities  as  is  mandated  by 
St.1995,  c.263; 

•  Develop  a  process  that  would  allow  the  AOTC  to  identify  any  and  all  deviations 
from  the  provisions  of  the  Revised  IT  Plan  filed  with  the  Required  Entities  on 
September  5,  2001; 

•  Develop  and  adopt  an  appropriate  definition  of  costs,  as  suggested  by  the  AOTC's 
Legal  Office  in  May  of  1996,  related  to  administration  and  implementation  that 
would  capture  the  full  cost  of  the  AOTC's  Project  Office,  the  function  and 
purpose  of  which  should  be  limited  to  the  management,  supervision  and  actual 
fulfillment  of  the  bond  bill's  stated  purpose.  In  addition,  costs  related  to  the 
administration  of  the  oversight  committees  and  sub-committees  as  well  as  any 
consultant  contract  costs  associated  with  the  discharge  of  the  functions  of  the 
oversight  committees  and  sub-committees. 
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Chapter  2: 


Management  of  the  Project 


Summary  of  Findings: 


1 .  The  AOTC  did  not  hire  a  project  manager  until  the  Spring  of  2000; 

2.  The  AOTC  failed  to  adhere  to  the  management  procedures  and  policies 

ESTABLISHED  FOR  THE  PROJECT; 

3.  The  AOTC  failed  to  follow  the  recommendations  of  the  project 

ADVISORY  AND  OVERSIGHT  COMMITTEES. 

4.  The  AOTC  failed  to.  independently  verify  the  technical  aspects  of 

DELIVERABLES  PROVIDED  BY  THE  SYSTEMS  INTEGRATOR  AND  OTHER  VENDORS. 

Introduction: 


"The  problem  is  that  the  Trial  Court  has  no 
articulated  philosophy  of  management,  and  no 
management  plan.  When  you  think  about  it,  injecting  a 
large-scale  computerization  initiative  into  state  level 
court  administration  as  it  now  exists  seems  almost  crazy. 
The  oversight  that  will  inevitably  accompany  the 
expenditure  of  large  amounts  of  money  is  going  to 
highlight  state  level  administrative  weaknesses  as  never 
before." 

-  Court  Administrator 
1993 


In  a  Government  Technology  magazine  article  entitled  "Risky  Business:  Better 
Management  Key  to  Big  System  Development",  Tod  Newcombe  wrote  that  "While  the 
private  sector  quickly  latched  on  to  the  idea  that  big  computer  projects  required  skilled 
managers  who  had  the  discipline  to  control  system  requirements  and  user  expectations, 


102 


the  public  sector  has  continued  to  ignore  the  importance  of  project  management."      This 
necessity  for  strong  project  management  is  made  more  important  in  an  organization  that 

1  0"? 

has  historically  lacked  strong  institutional  management. 

Numerous  reports  over  the  past  twenty-five  (25)  years  have  highlighted  the 
management  deficiencies  of  the  Trial  Court.  104  The  autonomous  nature  of  its  seven  (7) 
Departments  and  the  inherent  power  of  the  respective  Chief  Justices  frustrates  attempts  at 


102  Government  Technology  Magazine,  May  2000 

103  Bureau  Interviews,  November  5,  2001. 

104 


Report  on  the  State  of  Massachusetts'  Courts,  Select  Committee  on  Judicial  Needs,  December  1976,  the 
so-called  "Cox  Report";  Justice  Endangered:  A  Management  Study  of  the  Massachusetts  Trial  Court,  Final 
Report,  April  16,  1991,  by  Harbridge  House,  Inc. 
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centralization.10''  As  stated  earlier,  the  1992  legislation  and  the  Bond  Bill  represent 
efforts  by  the  Legislature  to  centralize  information  systems  within  the  office  of  the  CJAM 
and  within  a  centralized  Trial  Court  Information  Technology  Department  (AOTC- 
ITD).106 

From  the  beginning  of  the  project,  some  Departments  of  the  Trial  Court  have 
expressed  different  visions  as  to  what  the  final  system  should  look  like,  how  it  should  be 
obtained,  how  it  should  be  managed,  and  who  should  be  involved  in  the  decision-making 

1  07 

process.       The  lack  of  a  shared  vision  for  the  project  has  resulted  in  project  delay  and 
increased  cost.108  While  the  CJAM  has  been  given  the  authority  by  the  Legislature  to 
direct  and  manage  IT  projects,109  each  CJAM  who  has  served  during  this  project  has  been 
reluctant  to  intercede  when  disputes  arise  in  order  to  expedite  the  project.110 

An  independent  consultant  from  MIT's  Sloan  School  of  Management  hired  in 
2000  to  provide  an  assessment  of  the  project  found  that  "decisiveness  and  clear  direction 
is  needed  at  key  times.  There  comes  a  time  when  further  learning  and  discussions  are  not 
only  unnecessary,  but  also  counterproductive.  Continual  revisiting  of  questions  and 
issues  will  not  only  slow  the  process,  but  will  invite  debate  and  resistance  in  any 
politically  charged  culture  where  there  are  differences  of  opinion."111  He  concluded  that 
even  though  the  CJAM  has  a  fragmented  position  of  power,  emphasis  had  to  be  placed  on 
"decisiveness."112 

The  AOTC  was  aware,  as  early  as  1993,  that  a  strong  management  structure  was 
essential  to  successfully  complete  the  IT  Project.  A  September  1993  assessment 
concluded  that  "the  Trial  Court  (not  divisions)  has  first  and  foremost  an  IS  (information 
system)  management  problem,  not  a  technical  or  IS  projects  problem.  There  is  no 
program  for  the  planning,  budgeting,  financing,  coordination,  standardization, 
integration,  management  and  control  across  all  divisions  of  the  Trial  Court."        The 
AOTC  was  advised,  "poor  management,  control  and  execution  of  a  good  plan  will 
produce  the  same  results  as  good  management,  control  and  execution  of  a  poor  plan."114 

Individuals  who  would  eventually  occupy  critical,  high-level  roles  in  the  project 
shared  this  negative  view  of  the  Trial  Court's  management  structure  and  philosophy.  For 
example,  the  AOTC's  CIO  stated  in  November  of  1993  that  the  "organization  does  not 


105  "Review  of  Recommendations  by  the  Project  Advisory  Board  to  Build  a  Statewide,  Comprehensive 
Case  Management  System,"  Cyrus  F.  Gibson,  May  10,  2000. 

106  St.1995,  c.263;  and,  St.1992,  c.133,  §37. 

107  Court  Administrator  Letter,  June  16,  1995. 

108  Small  Claims  BasCOT  Change  requests,  June  of  1999. 

109  St.1992,  c.133,  §37. 

110  Bureau  Interview,  October  4,  2001;  "Review  of  Recommendations  by  the  Project  Advisory  Board  to 
Build  a  Statewide,  Comprehensive  Case  Management  System,"  Cyrus  F.  Gibson,  May  10,  2000. 

1.1  "Review  of  Recommendations  by  the  Project  Advisory  Board  to  Build  a  Statewide,  Comprehensive 
Case  Management  System,"  Cyrus  F.  Gibson,  May  10,  2000. 

1.2  Id. 

113  Hoggshead  +  Knapp  Master  Plan  at  2-2,  September  of  1993. 

114  Id.  at  2-3. 
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have  a  clear  idea  of  how  to  attack  management  issues."  He  concluded  that,  "the 
organization  does  not  enforce  its  own  rules.  There  is  no  evidence  of  accountability.115" 

With  this  management  assessment  as  a  backdrop,  the  AOTC's  consultant,  the 
Point  Group,  proposed  the  creation  of  a  temporary  Project  Manager  who  would  be  given 
the  responsibility  of  managing  the  project  in  coordination  with  the  AOTC-ITD.1 16    This 
position,  however,  was  neither  created  nor  filled  until  the  Spring  of  2000.  By  the  time 
the  Project  Manager  was  hired,  nearly  five  years  had  passed  since  the  passage  of  the  bond 
bill.117 

Findings: 

1 .         The  AOTC  did  not  hire  a  project  manager  until  the  Spring  of  2000. 

From  the  passage  of  the  Bond  Bill  through  the  Spring  of  2000,  the  CIO  for  the 
AOTC  performed  de  facto  project  management.1 18  He  performed  these  duties  while 
simultaneously  serving  as  the  CIO  for  the  AOTC.119  The  former  Executive  Director  of 
the  AOTC,  the  Chair  of  the  Project  Advisory  Board  (PAB),  and  the  former  Project 
Manager  all  stated  that  the  CIO  could  not  realistically  perform  all  of  the  duties  relative  to 
the  IT  Project  and  the  day-to-day  operations  of  the  AOTC-ITD.120  Notwithstanding  the 
awareness  that  a  separate  Project  Manager  should  be  hired,  a  Project  Manager  was  not 
hired  until  2000. 121 

When  a  Project  Manager  was  finally  hired  in  2000,  a  formal  delineation  of 
responsibilities  between  the  Project  Manager  and  the  CIO  were  apparently  never  drafted 
nor  adopted.  As  such,  neither  person  could  definitively  state  the  precise  parameters  of 
their  respective  authority.       It  is  apparent,  however,  that  the  Project  Manager  did  not 
have  an  unfettered  ability  to  directly  report  to  the  CJAM,  without  communicating  through 
the  CIO.      This  arrangement  was  not  conducive  to  strong  and  decisive  project 
management  because  there  was  no  recognized  person  within  the  AOTC  with  the  ability 
or  authority  to  exercise  strong  project  control.124  Because  the  lines  of  authority  were  not 
clearly  marked,  the  Project  Manager  also  believed  that  she  was  prevented  from  making 


115  Personnel  Interviews,  November  3,  1993. 

116  Organization  and  Management  Memorandum,  October  5,  1995. 

117  The  Project  Manager  hired  in  the  Spring  of  2000,  left  in  July  of  2001. 

118  Bureau  Interviews,  August  9,  2001,  and  October  4,  2001 . 

119  CJAM  Remarks,  February  14,  1996.  The  CJAM,  in  1996,  stated,  "the  director  of  the  Information 
Technology  Department,  will  be  responsible  for  the  day-to-day  oversight . .  .of  the  project." 

120  Bureau  Interviews,  August  2,  2001,  September  12,  2001,  and  October  4,  2001. 

121  The  CJAM  of  the  AOTC  was  informed  in  1996  that  a  Project  Manager  should  be  hired  to  oversee  the 
project.  Similarly,  the  Chair  of  the  PAB  informed  the  CJAM  on  numerous  occasions,  as  early  as  1996,  and 
then  as  Chair  of  the  PAB  between  1998  and  2000  that  a  Project  Manager  was  required  for  the  project.  A 
Project  manager  selection  process  was  finally  undertaken  in  1999.  At  that  time,  a  number  of  applicants 
were  interviewed,  including  the  CIO  of  the  AOTC-ITD.  The  review  committee  recommended  that  no 
person,  including  the  CIO,  be  hired  as  the  project  manager.  The  Chair  of  the  PAB  stated  that  none  of  the 
applicants  had  the  requisite  experience  or  managerial  skills  to  be  hired  as  the  Project  manager.  Action  was 
deferred  for  one  year. 

122  Bureau  Interviews,  October  4,  2001,  and  November  5,  2001. 

123  Id. 

124  Bureau  Interview,  November  5,  2001. 
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decisions  that  would  have  a  budgetary  impact.125  According  to  the  Chair  of  the  PAB,  the 
Project  Manager  was  not  invited  to  weekly  meetings  between  the  CJAM,  the  CIO  and  the 
AOTC's  Executive  Director  even  though  project  decisions  were  being  made  at  those 
meetings.126 

The  hiring  of  a  Project  Manager  in  the  Spring  of  2000  was  important,  but  this 
event  did  not  result  in  the  establishment  of  systemic  processes  to  guide  the  management 
and  the  decision-making  process  of  the  IT  Project.  Upon  taking  over  the  project  in  the 
Spring  of  2000,  the  Project  Manager  found  a  project  that  did  not  have  a  project 
management  plan;  no  risk  management  plan;  no  procurement  plan;  no  network 
infrastructure  plan;  no  work  order  process;  no  change  order  process;  no  deliverables 
acceptance  process;  no  quality  assurance  process  and  no  internal  policies  to  guide  the 
overall  interrelationship  between  the  many  departments  of  the  Trial  Court,  the  project 
oversight  entities  and  other  external  entities.  More  troubling,  however,  was  the  fact  that 
the  Project  Manager  found  that  there  was  no  comprehensive  plan  for  the  IT  Project.127 

2.         The  AOTC  failed  to  adhere  to  the  management  procedures  and 

POLICIES  ESTABLISHED  FOR  THE  PROJECT. 

In  1996,  when  it  became  apparent  to  the  AOTC  that  an  experienced  entity  was 
required  to  assist  the  CIO  with  the  implementation  of  the  IT  Plan,      the  decision  was 
made  to  seek  the  services  of  a  "systems  integrator."  (integrator)    A  Request  for 
Responses  (RFR)  was  issued  with  the  understanding  that  the  integrator  would  act  "as  a 
general  contractor  for  implementation  of  the  IT  plan."       The  integrator  would  also  serve 
as  a  technical  consultant  on  the  project. 

On  August  20,  1997,  Deloitte  &  Touche  ("Deloitte")  was  selected.131  After  the 
selection,  a  contract  was  negotiated  whereby  Deloitte  would  be  paid,  primarily,  based 
upon  the  time  and  materials  expended  on  the  project.132  In  addition,  work  orders  and 
change  orders  that  were  developed  throughout  the  project  were  considered  part  of  the 
contract.133  Deloitte  adopted  a  Project  Procedures  Manual  (PPM)  that  "was  to  be 
employed  by  the  Deloitte  Consulting  project  management  to  ensure  successful  project 
management."134  The  PPM  would  serve  as  the  "master  project  plan  throughout  the  life  of 
the  project"  and  it  would  be  used  to  govern  all  work  orders,  change  orders,  contractual 
disputes,  quality  assurance  issues,  staffing  issues,  deliverable  acceptance  issues,  and 


125  Bureau  Interview,  October  4,  2001. 

126  Bureau  Interview,  October  4,  2001. 

127  Bureau  Interviews,  August  2,  2001,  and  August  9,  2001. 

128  See,  September  5,  2001  Legislative  Filing. 

129  IT  Implementation  Document  at  1,  April  3,  1996. 

130  Id.  at  2. 

131  See,  Systems  Integration  Services  Recommendation,  August  20,  1997.  Deloitte  was  selected  as  a  result 
of  their  Best  And  Final  Offer  (BAFO)  relative  to  the  estimated  cost  of  the  project.  Deloitte  estimated  that 
their  costs  relative  to  the  tracks  contained  within  the  IT  Plan,  the  Court  operating  System,  the  desk  top  roll- 
out, the  expansion  of  existing  systems,  the  network  implementation  plan,  and  system  support  and 
administration  would  be  in  the  amount  of  $12  million.  The  other  finalist  estimated  that  it  would  incur  costs 
in  the  amount  of  $25  million.  As  of  July  1,  2001,  Deloitte  has  incurred  costs  in  excess  of  $21  million. 

132  Systems  Integrator  Contract,  February  1998. 

133  Project  Procedures  Manual,  April  1,  1998. 

134  Project  Procedures  Manual,  April  1,  1998. 
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project  governance.    '  As  such,  this  document  would  govern  the  contractual  relationship 
between  the  AOTC  and  Deloitte. 

The  CIO  informed  the  Bureau  that  he  failed  to  adhere  to  the  procedures 
established  within  the  PPM.136  He  stated  that  the  process  was  too  cumbersome,  and 
therefore  was  not  being  followed,137  and  that  it  would  have  required  that  he  spend  all  of 
his  time  reviewing  the  work  orders  and  change  orders.138  As  such,  the  process  was 
dropped  and  no  process  appears  to  have  taken  its  place.139  This  failure  to  follow  (or  use) 
an  established  management  procedure  that  was  intended  to  "govern"  the  project  resulted 
in  a  project  without  structure,  form,  or  a  guiding  plan.140  Because  the  CIO  did  not  follow 
the  PPM  procedures,  Deloitte  was  allowed  to  perform  without  effective  oversight.141 

By  July  of  1999,  the  CIO  was  aware  that  the  failure  to  effectively  oversee  and 
manage  Deloitte  and  the  project  was  becoming  a  problem.142  As  evidence  of  this 
problem,  work  orders  were  not  being  completed  on  time,  or  for  the  amount  that  had  been 
estimated  by  Deloitte.       Similarly,  because  the  PPM  processes  were  not  being  followed, 
change  orders  were  not  being  thoroughly  reviewed  and  compared  with  the  original  work 
orders.144  The  CIO  complained  that  Deloitte  and  the  AOTC  needed  to  "agree  on  an  order 
mechanism  that  works  and  then  use  them."145  Again,  there  is  no  evidence  that  any 
attempts  were  made  to  utilize  the  PPM  process  or  any  other  management  process  through 
that  date  (1999).  The  CIO  emphasized  that  the  management  mechanisms  should  be  "put 
down  on  paper"  and  used  "diligently  and  consistently."       Concerns  about  poor 
management  and  oversight  were  exacerbated  by  fears  that  Deloitte  was  "grossly" 
underestimating  work  orders  and  change  orders.  147  Without  a  strong  management 
process  in  place,  there  was  no  way  for  the  CIO  to  hold  Deloitte  to  the  estimates  they 
provided.148  Without  a  full-time  project  manager  in  place,  the  CIO  was  required  to 
attempt  this  task  on  his  own  while  simultaneously  managing  the  AOTC's  day-to-day  IT 

149 

operations. 


135  Id. 

136  Bureau  Interview,  August  9,  2001. 

137  The  CIO's  copy  of  the  PPM  contained  a  notation  that  stated,  "How  do  we  plan  to  review  these  115 
pages." 

138  See,  Bureau  Interviews,  Bureau  Interviews,  September  12,  2001,  and  October  4,  2001.  This  statement 
reinforces  the  observations  by  the  former  Executive  Director  and  the  Chair  of  PAB  that  the  CIO  could  not 
simultaneously  fulfill  his  responsibilities  as  the  CIO  of  the  AOTC  and  act  as  the  project  manager  for  a  $75 
million  IT  project. 

139  Bureau  Interviews,  August  9,  2001 . 

140  PAB  Management  Memorandum,  January  25,  2001. 

141  Bureau  Interviews,  August  2,  2001,  and  October  4,  2001. 

142  Contract  Renewal  Letter,  July  23,  1999. 

143  Id. 

144  Bureau  Interview,  November  5,  2001.  During  an  interview  with  the  Bureau,  a  representative  from 
Deloitte  indicated  that  the  AOTC  did  not  have  a  person  who  had  the  authority  to  manage  the  project  in  a 
decisive  fashion  vis-a-vis  the  individual  departments  of  the  AOTC.  As  will  be  discussed,  infra,  throughout 
the  project,  individual  departments  of  the  AOTC  would  insist  upon  changes  to  the  projects.  These  changes, 
if  not  controlled  by  the  AOTC,  result  in  added  costs  and  project  delays. 

145  Contract  Renewal  Letter,  July  23,  1999. 

146  Id. 

147  Id. 

148  Bureau  Interview,  October  4,  2001 . 

149  Id. 
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Therefore,  from  1996  through  the  Spring  of  2000,  the  day-to-day  operation  of  the 
project  was  left  to  the  AOTC's  CIO  who  was  unable  to  manage  and  oversee  the  activities 
and  planning  of  the  integrator.  As  a  result,  the  AOTC  was  unable  to  effectively  plan, 
budget,  or  to  stay  within  timelines  or  work  orders.' c 
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Even  though  the  CIO  was  unable  to  adhere  to  the  PPM  procedures,  advisory  and 
oversight  committees  had  been  established  to  assist  the  AOTC  with  the  implementation 
of  the  plan  and  the  management  of  the  project.  Here  too,  the  committees  proved 
ineffectual  due  to  poor  management,  poor  delineation  of  responsibilities,  and  an  over- 
reliant  emphasis  on  ad  hoc  task  management.  Notwithstanding  these  deficiencies,  their 
recommendations  were  often  ignored  or  overruled.151 

3 .    The  AOTC  failed  to  follow  the  recommendations  of  the  project 

ADVISORY  AND  OVERSIGHT  COMMITTEES. 

The  1 995  IT  Plan  called  for  the  creation  of  advisory  and  oversight  committees 
that  would  provide  guidance  to  the  Project  Manager  and  the  AOTC  relative  to  the  project. 
The  committees  would  assist  in  directing  the  scope  of  the  project  and  in  assessing  the 
needs  and  requirements  of  the  departments  of  the  Trial  Court  and  the  external  entities  that 
would  interface  with  the  comprehensive  system. 

The  original  advisory  and  oversight  committee  was  the  Information  Technology 
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Advisory  Committee  (IT  AC).       As  originally  structured,  IT  AC  was  given  direct  access 
to  the  CJAM.154  In  the  words  of  the  CJAM,  IT  AC  was  created  to  review  the  plans  and  to 
ensure  that  project  efforts  were  on  target.       In  addition,  IT  AC  was  charged  with 
promulgating  policies  and  procedures  for  the  project.156 

In  this  oversight  role,  the  original  co-chairs  of  IT  AC  sought  to  staff  the  committee 
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and  to  establish  formal  policies  and  procedures  for  the  project.       The  co-chairs  also 
believed  that  the  plan  that  had  been  filed  with  the  legislature  in  December  of  1995  was 
inadequate  and  that  a  more  definitive  strategic  plan  was  required  that  would  include  an 
inventory  of  existing  systems;  current  processes  and  data  used  throughout  the  Trial  Court; 
paths  followed  by  other  court  systems  in  automation;  trends  in  IT;  and,  a  schedule  of 
anticipated  actions  by  the  AOTC  necessary  to  implement  the  strategic  plan.       The  co- 
chairs  of  IT  AC  believed  that  a  strategic  plan  was  necessary  to  provide  a  reference  and 
planning  document  against  which  "all  questions  maybe  referred  and  answered."15' 


50  Id. 

51  Bureau  Interviews,  September  28,  2001,  and  October  4,  2001;  and,  PAB  Management  Memorandum, 
January  25,  2001. 

52  1995  IT  Plan,  December  of  1995. 
53 ITAC  was  formed  in  1996. 

54  Organization  and  Management  Memorandum,  October  5,  1995. 

55  CJAM  Remarks,  February  14,  1996. 

56  Organization  and  Management  Memorandum,  October  5,  1995. 

57  CIO  Letter  Regarding  ITAC  Co-Chair  Meeting,  December  21,  1996. 

58  Id. 

59  Id. 
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IT AC's  efforts  to  establish  a  concrete  strategic  plan  by  which  it  could  ensure  that 
project  "efforts  are  on  target"160  were  met  with  resistance  and  skepticism  by  the  AOTC- 
ITD.161  The  AOTC-ITD  believed  that  ITAC  was  going  to  be  used  to  benefit  the 
individual  departments  of  the  Trial  Court  to  the  detriment  of  centralized  IT  control.162 
This  is  another  example  of  the  inherent  management  difficulties  facing  the  Trial  Court  as 
a  body.  Notwithstanding  the  CJAM's  February  1996  statement  to  Department  Chief 
Justices,  Department  Court  Administrators,  and  Commissioners  that  the  project  was 
going  to  be  controlled  through  the  AOTC,  the  AOTC-ITD  believed  that  the  individual 
departments  of  the  Trial  Court  were  attempting  to  usurp  control  of  the  project  from  the 
AOTC. 

In  remarks  prepared  for  him,  the  CJAM  stated  to  the  representatives  from  the 
individual  departments  of  the  Trial  Court: 

"Your  role  in  this  process  is  critical.  I  need  your  constructive  participation  to 
ensure  that  the  business  requirements  of  the  Trial  Court  are  met.  I  welcome  your 
input  and  ideas  on  how  we  can  improve  efficiency  and  quality.  But  let  me  be 
clear  on  one  final  point.  My  office  and  the  Information  Technology  Department 
are  responsible  for  the  technology  decisions.  The  business  of  your  department  is 
your  responsibility.  The  design,  selection  and  implementation  of  technologies  to 
enable  you  to  more  effectively  and  efficiently  conduct  that  business  is  the 
responsibility  of  the  Administrative  Office." 

This  de  facto  demarcation  of  authority  and  control  has  yet  to  be  resolved.164  As  a  result 
of  the  real  or  perceived  struggle  between  the  AOTC  and  the  Trial  Court  departments  for 
the  control  of  the  project,  any  advice  and  oversight  that  was  rendered  by  ITAC  was 
viewed,  in  some  instances,  with  suspicion.165  Thus,  by  late  December  1996,  the 
management  of  the  IT  Project  began  to  fracture.166  This  is  prior  to  the  release  of  any 
substantial  bond  funds  for  the  IT  Project. 

"By  the  late  summer  of  1998  ITAC  and  its  subcommittees  had  stopped 
meeting."168  The  assessment  of  individuals  who  served  on  ITAC  was  that  it  was  too 
cumbersome,  had  no  real  planning  or  oversight  authority,  and  was  consequently 
ineffectual.169  Therefore,  project  planning  and  oversight  rested  solely  with  the  CIO  and 
the  AOTC-ITD.170 


160  Id. 

161  CIO  Letter  Regarding  ITAC  Co-Chair  Meeting,  December  21,  1996. 

162  Id.  "District  Court  and  ...  are  about  to  take  the  $75  million  to  town  for  whatever  this  committee  cooks 
up. . .  I  cannot  contemplate  at  this  point  going  through  another  round. . .  during  which  the  AODC  gives  us  an 
explanation  of  why  the  Brockton  system  answers  all  the  prayers  of  every  court." 

163  CJAM  Remarks,  February  14,  1996. 

164  Bureau  Interviews,  August  2,  2001,  August  9,  2001,  September  12,  2001,  and  October  4,  2001. 

165  CIO  Letter  Regarding  ITAC  Co-Chair  Meeting,  December  21,  1996. 

166  See,  "Review  of  Recommendations  by  the  Project  Advisory  Board  to  Build  a  Statewide,  Comprehensive 
Case  Management  System,"  Cyrus  F.  Gibson,  May  10,  2000. 

167  Excel  Spreadsheet,  Appropriation  0330-0951,  June  20,  2001,  Office  of  the  Comptroller  at  the  request  of 
the  Bureau. 

168  See,  September  5,  2001  legislative  filing  at  Section  3,  page  16,  17. 


169 
170 


Bureau  Interviews,  September  28,  2001,  and  October  4,  2001. 

Bureau  Interviews,  September  12,  2001,  September  28,  2001,  and  October  4,  2001. 
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In  October  of  1998,  Deloitte  presented  a  project  oversight  structure  to  the 
AOTC.       The  proposed  project  oversight  structure  recommended  the  creation  of  a 
Project  Executive  Committee  (PEC)  and  a  Project  Advisory  Board  (PAB).172  On  the 
proposed  organizational  chart,  the  PEC  had  direct  access  to  the  Chief  Justice  and  no  other 
entities.  The  PAB,  however,  was  subordinate  to  and  had  direct  access  only  to  the  AOTC 
IT.173 

The  PEC  was  required  to  review  the  project  status  on  a  quarterly  basis  and  advise 
the  CJAM  based  on  recommendations  from  the  project  management  team.174  The  PAB 
was  to  be  made  up  of  a  cross-section  of  representatives  from  the  Trial  Court's 
departments  and  was  charged  with  the  monthly  review  of  project  status,  and  advising  on 
"specific  project  issues.     "  As  with  ITAC,  certain  efforts  by  the  PAB  to  exercise 
planning  and  oversight  functions  were  met  with  suspicion  and  resistance  by  the  AOTC- 

ry    176 

For  example,  by  the  summer  of  1999,  the  PAB  was  attempting  to  advise  Deloitte 
regarding  the  deployment  of  interim  systems  developed  by  Deloitte.  * 7    While  this 
advisory  function  was  consistent  with  the  CJAM's  charge  to  the  PAB,      the  CIO 
believed  that  PAB's  efforts  undermined  the  responsibilities  and  authority  of  the  AOTC- 
IT.17'  This  resistance  would  become  more  evident  as  the  AOTC-IT  began  to  ignore  PAB 
requests  and  recommendations.180 


The  PAB  was  aware  of  and  concerned  with  the  lack  of  a  management  structure 
to  the  project.181  Consistent  with  its  charge  to  advise  the  AOTC-IT,  the  P^ 

sought  to  formulate  policies  and  procedures  that  would  allow  it  to  analyze  specific 

project  issues  and  to  prcn 

Chair  of  the  PAB  stated: 


relative  to  the  project.181  Consistent  with  its  charge  to  advise  the  AOTC-IT,  the  PAB 

ific 
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project  issues  and  to  provide  effective  oversight  on  planning  and  budget  issues.       As  the 


"A  project  of  the  complexity  and  enormity  as  is  the  development  of 
MassCourts  cannot  be  managed  by  the  seat  of  our  collective  pants. . .  At 
present  there  is  no  structured  collaboration  and  task  management  in  use  by 
the  IT  Department  or  if  there  is  the  ...  committee  has  not  been  informed 
about  it.  Even  if  there  are  aspects  of  group  management,  it  is  ad  hoc  and 
isolated."183 


171  Proposed  Project  Oversight,  October  21,  1998. 

172  Id. 

173  Id. 

174  Proposed  Project  Oversight,  October  21,  1998. 

175  Proposed  Project  Oversight,  October  21,  1998. 

176  CIO  Letter  to  Executive  Director,  August  1 1,  1999. 

177  Id. 

178  See,  September  5,  2001  Legislative  Filing  at  Section  3,  page  17,  note  12.  "the  PAB  ...  is  responsible  for 
making  recommendations  regarding  the  project  approach,  organization  and  priorities." 

179  CIO  Letter  to  Executive  Director,  August  11,  1999. 

180  Bureau  Interview,  October  4,  2001 . 

181  PAB  Management  Memorandum,  January  25,  2001. 

182  Bureau  Interview,  October  4,  2001 . 

183  PAB  Management  Memorandum,  January  25,  2001. 
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The  PAB  believed  that  the  "leaders  and  managers  at  every  level  of  the  project  must  have 
at  their  command  current  information  regarding  the  status  of  tasks  under  their 
jurisdiction,  easy  access  to  all  project  documents,  and  straightforward  methods  of 
communication. 
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As  a  result  of  the  PAB's  efforts  to  initiate  oversight  and  planning  activities,  its 
committee  on  Budget  and  Planning  requested  of  the  AOTC-IT  comprehensive  budgetary 
and  planning  documents  that  would  allow  the  PAB  to  assess  the  status  of  the  project.185 
Despite  numerous  requests,186  the  AOTC-IT  was  either  unwilling  or  unable  to  provide  the 
PAB  with  the  analysis  requested.18    As  of  October  5,  2001,  the  PAB  had  not  yet  received 
a  comprehensive  budgetary  and  planning  document  that  would  allow  it  to  provide 
effective  oversight  and  advice  for  the  project.188 

Similarly,  in  October  of  2000,  the  PAB  recommended  that  modifications  being 
made  to  the  interim  systems189  developed  for  the  departments  of  the  Trial  Court  be 
stopped  as  of  December  31,  2000. 190  This  recommendation  was  consistent  with  the 
budgetary  assumptions  provided  by  Deloitte  stating  that  it  was  unlikely  that  the 
comprehensive  integrated  system  could  be  developed  and  all  personnel  trained  for  the 
$75  million  unless  further  development  on  the  interim  systems  was  terminated.191 
Notwithstanding  this  analysis  by  Deloitte  and  the  recommendation  from  the  PAB,  further 
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modification  of  the  interim  systems  was  not  stopped.       Given  Deloitte' s  budgetary 
assessment,  the  failure  to  stop  the  development  of  the  interim  systems  could  jeopardize 
the  ability  of  the  AOTC  to  complete  the  project  for  the  amount  of  money  allocated. 
The  decision  to  ignore  the  PAB  request,  to  the  knowledge  of  the  Bureau,  was  not 
supported  by  any  budgetary  or  planning  analysis.194  If  the  analysis  existed,  neither  the 
PAB  nor  the  Bureau  was  provided  with  the  same. 
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Lastly,  the  PAB  directed  the  AOTC-ITD  and  Deloitte  to  include  a  Network 
Infrastructure  Design  within  an  RFR  that  is  to  be  released  for  the  MassCourts  system. 
The  AOTC-ITD  has  simply  failed  to  include  this  provision  within  the  RFR  and  did  not 
offer  the  PAB  a  rationale  for  the  failure.197 


184 


PAB  Management  Memorandum,  January  25,  2001 . 


185  Bureau  Interview,  October  4,  2001 ;  and,  Letter  to  Bureau,  October  5,  2001 . 

186  Bureau  Interview,  October  4,  2001.  The  Budget  and  Planning  subcommittee  of  the  PAB  made  the 
requests.  The  Chair  of  that  subcommittee  has  recently  resigned. 

187  Bureau  Interview,  October  4,  2001. 

188  Letter  to  Bureau,  October  5,  2001. 

189  For  a  discussion  of  the  interim  systems,  see  Chapter  2,  Finding  4,  infra. 

190  Assumptions  and  Considerations  for  PAB  Phasing  Recommendation,  October  23,  2000. 

191  Deloitte  Budget  Scenario  Planning,  January  1999;  Bureau  Interview,  November  5,  2001. 

192  Bureau  Interview,  October  4,  2001 . 

193  See,  Deloitte  Budget  Scenario  Planning,  January  1999;  and,  discussion  at  Chapter  3,  infra. 

194  Bureau  Interview,  August  9,  2001 . 

195  Bureau  Interview,  October  4,  2001. 

196  Id. 

197  Id. 
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The  AOTC's  resistance  to  committee  oversight  and  recommendations  has 
undermined  the  credibility  of  the  committees  and  negatively  impacted  the  desire  of  key 
participants  to  continue  on  this  project.198 

4.    The  AOTC  failed  to  independently  verify  and  validate  the  technical 

ASPECTS  OF  DELIVERABLES  PROVIDED  BY  THE  SYSTEMS  INTEGRATOR  AND 
OTHER  VENDORS. 

The  MIT  Sloan  School  of  Management  consultant  hired  by  the  AOTC  in  May  of 
2000  also  recommended  that  the  AOTC  contract  with  an  entity  to  independently  validate 
and  verify  technical  work  performed  on  the  project.199  This  requirement  was  especially 
crucial  given  Deloitte's  expanded  role  since  becoming  the  integrator.  As  the  integrator, 
Deloitte  was  required,  among  other  duties,  to  act  as  the  technical  consultant  to  the 
AOTC-IT.    '    Since,  Deloitte  was  responsible  for  much  of  the  technical  work  on  the 
project,  another  entity  would  be  required  to  verify  and  validate  Deloitte's  technical 
work.201 

In  1997,  the  AOTC  decided  that  interim  systems  had  to  be  developed  and 
deployed  throughout  the  Trial  Court  so  that  staff  at  the  court  level  would  experience  the 
benefits  of  computerization.       The  AOTC  believed  that  a  "reasonable  series  of  changes 
and  improvements. . .  should  permit  us  to  leverage  the  money  and  time  already  invested 

TAT 

into  the  Warrant  Management  System  (WMS)      into  a  consistent  transitional  system 
bridging  our  current  circumstances  to  the  day  when  a  more  carefully  selected  system 
becomes  available  for  deployment."204  By  July  of  1998,  however,  the  decision  to  use 
WMS  as  a  "transitional  system"  expanded  into  a  decision  to  "develop  interim 
systems."205  One  of  these  interim  systems  was  BasCOT.206 

BasCOT,  at  a  very  minimum,  was  to  provide  the  participating  Departments      of 
the  Trial  Court  with  the  same  level  of  functionality  that  existed  with  the  applications  then 
being  utilized  by  those  courts.  In  April  of  1999,  Deloitte  estimated  that  the  application 
would  be  deployed  statewide  by  December  of  1999.208  As  of  January  2001,  BasCOT  had 


198  Bureau  Interview,  October  4,  2001. 

199  See,  "Review  of  Recommendations  by  the  Project  Advisory  Board  to  Build  a  Statewide,  Comprehensive 
Case  Management  System,"  Cyrus  F.  Gibson,  May  10,  2000. 

200  IT  Implementation  Document  at  page  2,  April  3,  1996. 

201  See,  "Review  of  Recommendations  by  the  Project  Advisory  Board  to  Build  a  Statewide,  Comprehensive 
Case  Management  System,"  Cyrus  F.  Gibson,  May  10,  2000;  Letter  to  CJAM,  January  23,  2001 . 

202  IT  Implementation  Initiative,  January  7,  1997;  Presentation  to  SJC,  June  28,  2000;  Bureau  Interview, 
August  2,  2001. 

203  Id.  The  Warrant  Management  System  (WMS)  was  an  existing  application  that  had  been  developed  and 
deployed  to  departments  of  the  Trial  Court  prior  to  the  project. 

204  IT  Implementation  Document,  April  3,  1996. 

205  See,  September  5,  2001  Legislative  Filing  at  Section  3,  page  13. 

206  Basic  Court  Operating  Tools. 

207  Bureau  Interview,  October  4,  2001 .  The  Superior  Court  Department  did  not  participate  in  the  BasCOT 
interim  system  because  the  existing  system,  Forecourt  Vision,  provided  a  greater  level  of  functionality  than 
BasCOT  would  provide.  As  an  example,  the  interviewee  believed  that  Forecourt  Vision's  calendaring 
capabilities  would  not  be  available  in  BasCOT. 

208  District  Court-BasCOT  Development  Assumptions,  April  1999. 
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not  been  fully  deployed;209  and,  it  was  then  anticipated  that  BasCOT  would  be  deployed 
by  the  end  of  September  200 1.210 

As  of  November  2001,  this  interim  system  application  had  not  yet  been  fully 
deployed  even  though  four  years  had  passed  from  the  initiation  of  the  application 
development.21 '  One  of  the  difficulties  with  the  full  deployment  of  BasCOT  was 
summarized  by  the  AOTC's  Project  Manager  in  October  of  2000,212  when  she  stated, 
"there  is  a  risk  that  BasCOT  will  not  be  deployed  beyond  the  twenty-five  District  Court 
divisions  if  the  speed  of  the  application  is  found  to  be  inadequate  over  a  56K  line.  It  may 
be  either  cost  prohibitive  or  capacity  prohibitive  to  upgrade  the  remaining  courts  to  faster 
line  speeds."  13 

Essentially,  an  interim  system  has  been  designed,  developed  and  partially 
deployed  without  any  prior  assessment  as  to  the  speed  of  the  BasCOT  application  or  the 
line  capacities  of  each  court.214  Prior  to  the  initiation  of  this  interim  system,  the  AOTC 
and  Deloitte  failed  to  prepare  a  technical  analysis  of  the  capabilities  of  each  court  and 
failed  to  perform  an  analysis  of  the  appropriate  line  speed  required  for  the  proper 
operation  of  the  BasCOT  application.  As  of  October  2000,  the  interim  civil  system  was 
only  available  at  twenty- five  courts  that  handled  58%  of  the  civil  caseload.215  As  such, 
three  years  into  the  BasCOT  component  of  the  project,  forty- four  courts  and  42%  of  the 
civil  caseload  could  not  benefit  from  the  interim  system,216  even  though  the  stated 
purpose  for  developing  BasCOT  and  other  interim  systems  was  "the  very  real  need  to 
make  technology  relevant  to  a  large  portion  of  the  Judiciary." 

Had  a  technical  consultant  been  in  place  to  analyze  the  development  and 
deployment  assumptions  of  BasCOT  in  light  of  the  stated  goals  of  providing  the  benefits 
of  computerization  throughout  the  courts,  it  would  have  been  obvious  that  the  existing 
network  capacities  throughout  the  Departments  of  the  Trial  Court  would  not  support  full 
deployment.  Because  this  analysis  was  not  prepared,  the  Bureau  cannot  credit  the 
AOTC's  assertion  that  "When  there  is  a  need  to  increase  capacity  in  the  network  it  can  be 
accomplished,  within  reasonable  limits,  rapidly  and  easily." 

Recommendations : 


To  improve  the  management  and  structure  of  the  IT  project  and  to  help  ensure  that  the 
project  is  completed  on  time  and  within  budget,  the  AOTC  should  take  the  following 
actions: 


209  Interim  Systems  Schedule,  January  16,  2001. 

210  Id. 

211  Bureau  Interview,  November  5,  2001. 

212  Potential  Drawback  to  BasCOT  Rollout,  Project  Manager  File,  October  2000. 

213  Id. 
2,4  Id. 

215  Potential  Drawback  to  BasCOT  Rollout,  Project  Manager  File,  October  2000. 

2,6  Id. 

217  See,  September  5,  2001  Legislative  Filing  at  Section  3,  page  13. 

218  Id.  at  13,  14. 
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Suspend  further  work  on  the  design,  development  and  deployment  of  MassCourts 
until  the  AOTC  has  created  the  management  structure  and  established  the  policies 
and  procedures  required  for  the  management  of  the  project; 

•  Develop  an  effective  project  management  plan  before  releasing  the  RFR  for 
MassCourts  so  that  the  AOTC  can  effectively  control  and  monitor  the  progress  of 
the  project; 

•  Establish  policies  that  ensure  that  the  individual  departments  of  the  Trial  Court 
understand  the  CJAM's  statutory  role,  responsibility,  and  authority  relative  to  the 
project; 

•  Hire  a  Project  Manager  who  has  appropriate  managerial  skills  and  experience 
managing  projects  of  this  size; 

Establish  a  project  organizational  structure  where  the  Project  Manager  reports 
solely  to  the  CJAM; 

•  Establish  a  project  organizational  structure  where  the  Chief  Information  Officer  of 
the  AOTC's  role  and  authority  is  relegated  to  advising  the  Project  Manager  on 
issues  related  to  the  IT  Project; 

•  Ensure  that  the  Project  Manager  has  a  clearly  delineated  and  understood  role, 
responsibility,  and  authority  relative  to  the  IT  Project; 

•  Ensure  that  the  Project  Advisory  Board  and  the  Project  Executive  Committee 
have  clearly  delineated  and  understood  roles,  responsibilities,  and  authority 
relative  to  the  IT  Project; 

•  Develop  written  management  policies  defining  the  Trial  Court's  project  structure, 
reporting  structure,  and  oversight  structure; 

More  specific  recommendations  with  respect  to  project  selection  and  project  control  are 
included  at  Exhibit  B. 
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Chapter  3 


An  Uncertain  Future 


Summary  of  Findings: 


1 .  The  new  Project  manager  does  not  have  a  detailed  plan  for 
the  daily  management  of  the  project. 

2.  The  AOTC  is  seeking  a  new  vendor  that  will  have  to  verify  the 

SUBSTANTIAL  WORK  PERFORMED  BY  THE  SYSTEMS  INTEGRATOR. 

3 .  The  AOTC  does  not  know  if  MassCourts  will  be  an  adaptation 
of  an  existing  commercially  available  system  or  if  it  will  be 
specifically  developed  for  the  trial  court 

4.  The  History  of  the  project  and  the  budget  scenarios  prepared 
by  the  systems  integrator  cast  doubt  as  to  whether  the 
project  will  be  completed  as  specified  within  the  scope 
document  for  $75  million. 


Introduction: 


"A  project  of  the  complexity  and  enormity  as  is  the 
development  of  MassCourts  cannot  be  managed  by  the 
seat  of  our  collective  pants. . .  At  present  there  is  no 
structured  collaboration  and  task  management  in  use  by 
the  IT  Department  or  if  there  is  the  . . .  committee  has 
not  been  informed  about  it.  Even  if  there  are  aspects  of 
group  management,  it  is  ad  hoc  and  isolated." 

-Chair  of  Project  Advisory  Board 

2001 


On  September  5,  2001,  the  AOTC  submitted  a  revised  project  plan  and  an 
assessment  of  the  status  of  the  project  to  the  Legislature.  The  revised  plan  calls  for  the 
completion  of  the  IT  Project  by  June  30,  2005.  In  addition,  the  AOTC  states,  "it  is 
anticipated  that  the  IT  Project  will  be  completed  within  that  budget  ($75,000,000), 
without  the  need  to  seek  additional  bond  funding." 


219 


The  Bureau  is  unable  to  credit  the  assertion  that  the  project  will  be  completed  by 
2005  or  that  it  will  be  finished  with  the  remaining  balance  of  $34.4  million  in 
unexpended  bond  funds. 


219 


See,  September  5,  2001  Legislative  Filing. 
220  Id. 
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The  Chair  of  the  PAB,  the  committee  presently  charged  with  budgetary  and 
planning  oversight  for  the  project,  indicated  that  neither  he  nor  the  Budget  and  Planning 
sub-committee  of  the  PAB  had  participated  in  the  revised  plan  or  the  budgetary  analysis 
that  was  prepared  for  the  legislative  filing.221  The  PAB  chair  volunteered  that  the  AOTC- 
IT  had  been  unable,  despite  numerous  requests,  to  provide  the  PAB  with  a 

Oil 

comprehensive  budgetary  analysis.       He  stated  that  he  was  unaware  of  any  cost  analysis 
or  budget  estimates  that  had  been  prepared  by  the  AOTC-IT.223 

The  preparation  of  the  revised  IT  Plan  and  assessment  without  the  benefit  of 
input,  advice  or  oversight  from  the  PAB,  the  committee  charged  with  these 
responsibilities,  is  evidence  that  the  management  problems  that  have  existed  since  the 
initiation  of  this  project  have  not  been  resolved.224 

Therefore,  based  on  the  history  of  the  IT  Project  and  the  uncertainty  that  faces  the 
selection  of  a  new  vendor,  it  is  doubtful  that  the  comprehensive  integrated  automated 
system  will  be  designed,  developed  and  deployed  for  the  budgeted  amount. 

1 .         The  new  Project  manager  does  not  have  a  detailed  plan  for  the  daily 
management  of  the  project. 

The  revised  IT  Plan  is  inadequate  as  "a  detailed  project  plan  that  a  project 
manager  would  use  to  closely  monitor  the  progress  of  the  project  on  a  periodic  basis  (at 
least  quarterly). . .  The  project  manager  would  need  to  check  compliance  with  milestones 
as  they  come  up,  the  tax  on  resources  as  they  occur,  monitor  and  correct  anticipated 
contingencies  promptly."225  According  to  the  Chair  of  the  PAB,  the  revised  IT  Plan 
submitted  to  the  Legislature  is  sufficient  for  purposes  of  the  legislative  filing 
requirements,  but  it  is  insufficient  for  purposes  of  day-to-day  management  of  the 
project.       This  comment  by  the  PAB  chair  mirrors  the  critique  offered  in  1996  by  the 
co-chairs  of  IT  AC  about  the  original  IT  Plan  filed  with  the  Legislature. 

In  addition,  the  new  Project  Manager     will  not  have  the  benefit  of  a 
comprehensive  management  plan  in  place  that  would  delineate  the  Project  Manager's 
responsibilities  and  authority  within  the  AOTC.  As  stated  earlier,  the  previous  Project 


221  Bureau  Interview,  October  4,  2001 

222  Id. 

223 


Id. 

224  Letter  to  the  Bureau,  October  5,  2001.  "Review  and  adoption  of  such  a  detailed  project  plan  is  critical  to 
any  policy  setting  board  such  as  the  PAB  since  it  sets  out  the  resource,  budgetary,  milestone,  staffing,  and 
technical  framework  within  which  the  policy  board  would  make  policy  decisions  affecting  the  project.  As  I 
indicated  during  my  interview,  except  for  the  proposed  project  plan  submitted  by  Deloitte  (which. .  .was  not 
adopted),  the  Project  Advisory  Board  has  not  reviewed  a  detailed  project  plan,  in  spite  of  frequent  requests 
for  such. 

225  Id. 

226  Id. 

227  CIO  Letter  Regarding  ITAC  Co-Chair  Meeting,  December  21,  1996. 

228  According  to  the  Revised  Plan,  the  new  project  manager  is  scheduled  to  begin  work  on  November  1, 
2001 .  The  project  has  not  had  a  project  manager  since  July  of  200 1 .  It  appears  that  a  Project  Manager  has 
not  yet  been  hired. 

35 


Manager  was  uncertain  as  to  the  extent  of  her  authority  vis-a-vis  the  CIO  and  the 
departmental  IT  departments. 

Also,  there  is  no  articulated  risk  management  plan  that  will  allow  the  Project 
Manager  to  gauge  the  project's  risks  by  evaluating  objectives,  identifying  areas  of 
uncertainty  and  formulating  a  cost  effective  strategy  for  dealing  with  risks.       There  are 
no  pre-determined  mitigation  procedures  to  deal  with  management  and  technical 
problems  as  they  arise. 

Finally,  the  required  skills  of  a  Project  Manager  for  this  IT  Project  have  been  the 
subject  of  dispute  within  the  AOTC.232  As  stated  earlier,  the  CIO  of  the  AOTC  was 
responsible  for  the  day-to-day  oversight  of  the  project  from  1996  through  the  Spring  of 
2000.23    It  was  the  belief  of  the  former  Executive  Director  of  the  AOTC,  that  technical 
skills  and  expertise  were  the  most  critical  skills  to  look  for  in  a  manager  for  the  IT 
Project.234  Expressing  a  contrary  view,  the  Chair  of  the  PAB  believed  that  a  Project 
Manager  should  be  conversant  in  technical  areas,  but  the  main  skill  set  should  be  in 
managing  human,  material  and  financial  resources.       It  is  not  apparent  to  the  Bureau 
that  this  philosophical  dispute  has  been  resolved  or  accounted  for  within  the  AOTC. 
As  was  reported  in  Government  Technology  magazine  in  May  of  2000,  "Technical 
heroes  had  the  role  of  project  manager  thrust  upon  them  by  their  bosses,  who  mistakenly 
believed  that  someone  who  knew  how  technology  worked  would  know  how  to  make  a 
technology  project  work. .  .Without  these  and  other  skills,  the  technical  hero,  suddenly 
promoted  to  IT  project  manager,  can't  control  the  outcome." 

In  short,  the  new  Project  Manager,  who  may  or  may  not  have  experience  managing  a 
project  of  this  scope,  will  be  stepping  into  the  position  without  the  necessary  supports  to 
facilitate  the  successful  management  of  the  IT  Project.  It  is  likely  that  the  new  Project 
Manager  will  spend  a  significant  amount  of  time  formulating  the  plans,  procedures,  and 
policies  that  are  required  to  successfully  manage  the  IT  Project.  While  this  is  occurring, 
actual  project  management  will  continue  on  an  ad  hoc  basis. 

2.         The  AOTC  is  seeking  a  new  vendor  that  will  certainly  have  to  verify 

THE  SUBSTANTIAL  WORK  PERFORMED  BY  THE  SYSTEMS  INTEGRATOR. 

Adding  to  the  uncertainty  that  will  be  faced  by  the  new  Project  Manager  is  the  fact 
that  Deloitte  will  not  design,  develop  or  deploy  the  comprehensive  integrated  computer 
system  required  by  the  bond  bill.238  This  fact,  perhaps  more  than  any  other,  throws  the 
AOTC's  assertion  that  the  project  will  be  completed  within  budget  into  doubt. 


229  Bureau  Interviews,  August  2,  2001,  October  4,  2001,  November  5,  2001. 

230  Bureau  Interview,  August  2,  2001,  and  August  9,  2001. 

231  Id. 

232  Bureau  Interviews,  September  12,  2001,  October  4,  2001. 

233  CJAM  Remarks,  February  14,  1996.  The  CJAM,  in  1996,  stated,  "the  director  of  the  Information 
Technology  Department,  will  be  responsible  for  the  day-to-day  oversight . .  .of  the  project." 

234  Bureau  Interview,  September  12,  2001. 

235  Bureau  Interview,  October  4,  2001 . 

236  See,  September  5,  2001  Legislative  Filing. 

237  "Risky  Business:  Better  Management  Key  to  Big  System  Development,"  Government  Technology  (May 
2000). 

238  Bureau  Interview,  October  4,  2001 . 
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As  the  integrator  since  February  of  1998,  Deloitte  has  come  to  know  the  Trial 
Court,  in  the  words  of  the  Chair  of  the  PAB,  "better  than  we  know  ourselves."239 
Deloitte's  representatives  and  management  conducted  and  oversaw  the  compilation  of 
system-wide  business  requirements  and  case  use  descriptions.240  In  addition,  Deloitte 
prepared  a  comprehensive  System  Scope  document  that  described  the  comprehensive 
system,  now  known  as  MassCourts?41 

Prior  to  beginning  the  design  for  the  construction  of  the  MassCourts,  the  AOTC 
and  the  Project  Manager  concluded  that  the,  primarily,  time  and  materials  integrator 
contract  would  not  be  appropriate  for  the  construction  of  a  system  of  this  size  and 
scope.       The  AOTC  and  Deloitte  initiated  contract  negotiations  for  a  contract  for  the 
construction  of  MassCourts.       As  negotiations  progressed,  it  became  evident  to  the 
negotiators  for  the  AOTC  that  negotiating  a  fixed  price  contract  would  be  problematic.244 
In  addition  to  differences  related  to  fixing  a  price,  other  issues  included,  but  were  not 
limited  to,  the  precise  role  of  Trial  Court  software  design  groups,  the  appropriate 
technical  and  application  architecture,  the  appropriate  testing  mechanisms  and  process, 
ultimate  ownership  of  the  software,  and  questions  of  damages  and  limitations  on 
liability.245  Many  of  the  issues  that  were  raised  during  this  round  of  negotiations  were 
focal  points  of  contention  during  the  life  of  the  integrator  contract.246  Issues  involving 
testing,  acceptance  of  deliverables,  project  structure,  and  payment  had  been  long  standing 
problems  from  the  perspective  of  the  AOTC.247  As  such,  the  contract  negotiation  for 
MassCourts  was  an  opportunity  to  resolve  the  problems  that  had  developed  over  the  prior 
three  years. 

By  April  of  2001,  the  AOTC  decided  to  cease  further  negotiation  with  Deloitte 
and  seek  a  new  vendor  for  MassCourts.249  Prior  to  ceasing  the  negotiations  and  seeking  a 
new  vendor,  however,  the  AOTC  did  not  analyze  nor  assess  the  cost  in  time  and  money 
that  would  be  involved  in  seeking  a  new  vendor.       A  new  vendor  will  require  an  RFR 
process      that  has  been  estimated  to  last  between  10  to  16      months  and  cost  between  $2 


239 


Bureau  Interview,  October  4,  2001 . 


240  Deloitte  System  Scope  Document,  August  2000;  Bureau  Interviews,  August  2,  2001,  August  9,  2001, 
September  28,  2001,  and  October  4,  2001.  These  documents  provided  a  detailed  description  of  the  tasks 
and  daily  functions  of  the  various  courts  that  make  up  the  departments  of  the  Trial  Court. 

241  Id. 

242  Bureau  Interview,  August  2,  2001. 

243  Id.  The  AOTC  hired  a  Bingham,  Dana  as  outside  counsel  to  negotiate  on  its  behalf. 

244  Bureau  Interviews;  Bureau  Interviews,  August  9,  2001,  September  12,  2001,  and  October  4,  2001 . 

245  Negotiations  Summary,  June  20,  2001. 

246  Deloitte  Contract  Renewal  Letter,  July  23,  1999. 

247  Id. 

248  Bureau  Interview,  August  2,  2001. 

249  Id.  The  Bureau  was  informed  that  Deloitte  was  notified  via  letter  of  the  "drop-dead"  date  of  April  13, 
2001  by  letter  of  Bingham,  Dana,  Counsel  to  the  AOTC. 

250  Bureau  Interview,  October  4,  2001. 

251  Deloitte  and  AOTC  are  drafting  the  RFR.  The  AOTC  has  stated  that  Deloitte  will  not  be  allowed  to  bid 
on  the  RFR. 

252  Deloitte  Build  v.  Buy  Discussion  Document,  July  12,  1999;  and,  "Review  of  Recommendations  by  the 
Project  Advisory  Board  to  Build  a  Statewide,  Comprehensive  Case  Management  System,"  Cyrus  F. 
Gibson,  May  10,  2000. 
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to  $3  million.  "  In  addition,  it  would  be  advisable  to  have  an  outside,  disinterested  party 
review  the  RPR  to  make  sure  that  the  document  as  drafted  does  not  favor  or  disfavor  any 
party  that  may  bid  on  the  RFR.254 

After  the  RFR  process  has  concluded  and  a  new  vendor  has  executed  a  contract, 
the  new  vendor  will  seek  to  verify  the  business  requirements  and  scope  documents 
prepared  by  Deloitte.  " '  By  way  of  example,  when  Deloitte  took  over  as  the  Integrator,  it 
initiated  a  review  of  the  comprehensive  business  requirements  that  had  been  prepared  by 
the  AOTC  in  conjunction  with  the  Point  Group.       Deloitte  estimated  in  February  of 
1998  that  the  review  and  revision  would  be  complete  by  mid- June  1998.25    As  of  January 
1999,  however,  the  Business  Requirements  confirmation  project  was  not  complete  and 
Deloitte,  at  that  time  estimated  that  the  Business  Requirements  confirmation  process 
would  be  complete  by  the  end  of  1999.25    In  fact,  the  Business  Requirements 
confirmation  and  review  process  continued  through  2000. 259  The  revised  IT  Plan  does 
not  contemplate  this  confirmation  process,  nor  does  it  assign  a  cost  estimate  for  the 

260 

process. 

The  uncertainty  that  the  new  vendor  will  bring  to  the  project  is  exacerbated  by  the 
fact  that,  despite  two  full  years  of  debate  and  countless  recommendations,  the  AOTC 
does  not  know  whether  MassCourts  will  be  a  "build"  or  "buy  and  modify"  to  suit  the 
needs  of  the  Trial  Court.261 

3.         The  AOTC  does  not  know  if  MassCourts  will  be  an  adaptation  of  an 

EXISTING  COMMERCIALLY  AVAILABLE  SYSTEM  OR  IF  IT  WILL  BE  SPECIFICALLY 
DEVELOPED  FOR  THE  TRIAL  COURT 

In  its  revised  IT  Plan,  the  AOTC  states  that  the  "successful  vendor  will  propose  a 
system  that  can  provide  all  the  features  and  functionality  encompassed  within  the  Scope 
Document  operating  on  a  computing  platform  and  network  that  meets  or  exceeds  the 
reference  platform  that  the  RFR  will  include.  The  RFR  will  permit  responses  ranging 
from  outright  new  development  of  MassCourts  to  adaptation  of  an  existing  commercially 
available  case  management  system,  (emphasis  added)" 

The  decision  to  "build"  or  "buy  and  modify"  has  been  a  focal  point  of  dispute 
throughout  this  process.263  In  May  of  2000,  an  independent  consultant264  was  hired  to 
"review  the  process  utilized  and  the  information  considered"  by  the  PAB  in 


253  See,  September  5,  2001  Legislative  Filing.  The  AOTC  estimates  that  the  RFR  process  will  cost 
$872,000.  The  $2  to  $3  million  estimate  is  derived  from  an  analysis  prepared  by  Deloitte  in  1999,  although 
the  Deloitte's  Contract  Manager  claimed  that  it  would  cost  less. 

254  Bureau  Interview,  October  4,  2001. 

255  Bureau  Interview,  October  4,  2001. 

256  Presentation  to  ITAC,  February  26,  1998. 

257  Id. 

258  IT  Project  Road  Map,  January  10,  1999. 

259  Bureau  Interview,  August  2,  2001 . 

260  See,  September  5,  2001  Legislative  Filing. 

261  Id.;  and,  Bureau  Interviews,  September  28,  2001,  October  4,  2001,  and  November  5,  2001 . 

262  See,  September  5,  2001  Legislative  Filing. 

263  CJAM  Letter  to  SJC,  June  12,  2000;  Bureau  Interviews,  August  2,  2001,  August  9,  2001,  September  12, 
2001,  September  28,  2001,  and  October  4,  2001. 

264  Professor  Cyrus  Gibson,  MIT,  Sloan  School  of  Management. 
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recommending  the  development,  rather  than  purchase,  of  the  comprehensive  integrated 
system:  MassCourts265  In  his  report  the  consultant  assessed  the  risks  inherent  in  either 
approach  and  concluded  that  the  risks  inherent  when  purchasing  a  system,  in  general,  are 
fewer  than  the  risks  involved  in  building  a  system.266  In  detailing  some  of  the  inherent 
risks  under  either  approach,  the  consultant  indicated  that  the  risks  to  buying  involved: 

►  The  necessity  for  multiple  packages  or  packages  plus  custom  modules  to 
meet  all  of  the  required  functionality; 

►  The  probability  work  would  be  required  with  the  vendor  to  correct  "bugs" 
in  the  programs  that  would  undoubtedly  arise; 

►  The  realization  that  the  package  required  would  be  quite  large  in  order  to 
meet  the  needs  of  the  whole  system; 

►  The  need  for  interfaces  within  the  Trial  Court  and  to  external  entities 
would  increase  risk.267 

The  risks  to  building  a  system  involved: 

►  The  builder's  experience  with  the  technology  (i.e.,  the  programming 
language  and  operating  system  and  support  infrastructure  being  used); 

►  The  sheer  size  of  the  project  relative  to  the  builder's  experience; 

►  The  builder's  experience  working  with  problems  of  similar  "structure"  to 
the  one  being  assessed,  where  structure  means  the  complexity  of  the 
problem  being  automated. 

Notwithstanding,  the  general  rule  that  building  is  inherently  more  risky  than 
buying,  the  consultant  concluded  that  a  "build"  was  less  risky  than  at  the  beginning  of  the 
project  because  "Deloitte  and  the  IT  resources  of  the  Trial  Court  have  gained  experience 
with  the  technology."       The  CJAM  echoed  this  finding  when  she  stated  that  it  was  true 
that  the  consultant  assessed  the  technical  "risk  of  buying  as  slightly  less  than  building. 
However,  the  technical  risk  that  the  system  will  not  work  as  designed  is  a  risk  that 
Deloitte  will  assume.270  Deloitte' s  technical  experience  is  being  tested  now  and  we  will 
have  the  benefit  of  their  hands-on  experience  with  our  courts,  (emphasis  added)" 


265  "Review  of  Recommendations  by  the  Project  Advisory  Board  to  Build  a  Statewide,  Comprehensive 
Case  Management  System,"  Cyrus  F.  Gibson,  May  10,  2000. 

266  "Review  of  Recommendations  by  the  Project  Advisory  Board  to  Build  a  Statewide,  Comprehensive 
Case  Management  System,"  Cyrus  F.  Gibson,  May  10,  2000. 
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269  Id. 

270  Bureau  Interview,  November  5,  2001.  A  risk  that  Deloitte  was  not  financially  ready  to  assume  as  is  seen 


by  the  failed  negotiations. 

271  CJAM  Letter  to  SJC,  June  12,  2000. 


39 


As  mentioned  earlier,  Deloitte  will  not  be  allowed  to  bid  on  the  RFR  that  they  are 
assisting  the  AOTC  in  drafting.2' '  As  such,  Deloitte  would  be  precluded  from  being 
selected  as  the  vendor  for  MassCourts.213  Therefore,  it  would  appear  that  the  risks 
inherent  in  building  are  no  longer  mitigated  since  Deloitte' s  experience274  will  not  be 
utilized  as  was  referenced  by  the  consultant  and  the  CJAM.275 

The  "buy  and  modify"  versus  "build"  issue  divided  the  PAB  and  various 

'7*7 f\ 

departments  within  the  Trial  Court.       Throughout  the  decision-making  process,  strong 
opinions  developed  concerning  the  feasibility  of  the  two  approaches.277  Initially  the  PAB 
voted  in  favor  of  a  "build"  solution  in  July  of  1999.278  A  decision  was  made  to  revisit  the 
issues  and  from  August  of  1999  through  January  of  2000,  the  PAB  investigated  the 
"buy/build  options  without  the  assistance  of  Deloitte,  since  they  were  interested  in 
building  the  system."2  '  Finally,  in  May  of  2000  the  CJM  adopted  the  PAB's  decision  to 

lOA 

"build." Notwithstanding  that  seemingly  final  decision,  the  cessation  of  contractual 

negotiations  with  Deloitte  relative  to  MassCourts  has  resulted  in  the  AOTC,  once  again, 
revisiting  the  "buy  and  modify"  versus  "build"  decision.281     The  AOTC  has  given 
prospective  MassCourt  vendors  the  ability  to  propose  their  own  solution  to  the  "buy  and 
modify"  versus  "build"  decision.2 

Therefore,  there  is  a  great  deal  of  uncertainty  and  division  concerning  what 
MassCourts  will  look  like  and  how  it  will  be  developed.  It  is  highly  doubtful  that  the 
design  phase  of  MassCourts  will  be  completed  between  the  second  quarter  of  2002  and 
the  first  quarter  of  2003  as  anticipated  in  the  revised  IT  plan.       The  schedule  included 
within  the  revised  IT  Plan  assumes  that  the  Trial  Court,  as  an  institution,  will  be  able  to 
coalesce  behind  a  single  development  methodology  in  a  quick  and  decisive  fashion.  In 
fact,  the  "buy/build"  decision  will  be  a  focal  point  of  the  RFR  review  process  and  any 
contract  negotiations.284 

The  difficulties  facing  the  Trial  Court  through  the  selection  of  a  vendor  and  the 
concurrent  "buy/build"  decision-making  process  is  symptomatic  of  the  cultural  and 
institutional  management  problems  existing  within  the  Trial  Court.       In  his  "buy/build" 


272  Bureau  Interview,  October  4,  2001. 

273  This  is  the  case  unless  all  vendors  are  rejected  and  the  AOTC  resumes  negotiations  with  Deloitte. 

274  Bureau  Interview,  November  5,  2001.  Deloitte  stated  that  they  may  still  take  part  in  the  development  of 
MassCourts 

275  Bureau  Interviews,  September  28,  2001,  and  October  4,  2001. 

276  Framingham  Conference  Buy/Build  Comments,  February  8,  2000;  and,  District  Court  Buy/Build 
Comments,  February  28,  2000. 
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278  Massachusetts  Lawyer's  Weekly,  Trial  Court  Responds  to  Story  on  Computers,  September  18,  2000. 
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280  CJAM  Letter  to  SJC,  June  12,  2000. 

281  See,  September  5,  2001  Legislative  Filing;  and  Bureau  Interviews,  September  28,  2001;  and  October  4, 
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285  Bureau  Interview,  October  4,  2001 ;  and,  See,"  Review  of  Recommendations  by  the  Project  Advisory 
Board  to  Build  a  Statewide,  Comprehensive  Case  Management  System,"  Cyrus  F.  Gibson,  May  10,  2000. 
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assessment,  the  consultant  concluded  in  May  of  2000  that: 

"In  summary,  the  important  issue  is  not  whether  to  buy  or  build, 
but  to  gain  support  for  the  chosen  option,  ..."286 

The  emphasis  on  gaining  project  support  is  essential  in  light  of  the  comments  of  a 
department  level  court  administrator,  who  stated  in  February  of  2000, 

"The  Trial  Court's  IT  culture  -  its  instincts,  expectations,  habits,  practices, 
etc.  -  is  simply  not  adequate  to  support  building  a  system.  In  some  cases 
we  affirmatively  devalue  some  indispensable  qualities,  like 
communications.  It  isn't  just  that  we  communicate  poorly.  It  is  that  we 
affirmatively  value  non-communications.  It  is  difficult  to  be  optimistic 
about  overcoming  these  limitations."287 

In  light  of  these  perceptions,  the  ultimate  "buy/build"  decision  will  be  complicated, 
lengthy  and  costly.  The  inability  to  gain  department-level  support  for  the  ultimate  project 
a  full  six  years  after  passage  of  the  bond  bill  is  a  troubling  fact  and  a  reflection  of  the 
overall  management  of  the  project. 

While  efforts  have  been  taken  to  address  the  management  deficiencies  of  the 
project,  the  revised  IT  Plan,  which  was  neither  reviewed  nor  adopted  by  the  AOTC's 
project  planning  and  oversight  committee,  casts  doubt  on  the  AOTC's  commitment  to 
improved  management.       The  failure  to  internally  review  and  analyze  the  revised  IT 
Plan  or  to  have  planning  committee  input  raises  concerns  about  the  reliability  of  the 
revised  IT  Plan.290 

4.         The  History  of  the  project  and  the  budget  scenarios  prepared  by  the 
systems  integrator  cast  doubt  as  to  whether  the  project  will  be 
completed  as  specified  within  the  scope  document  for  $75  million. 

On  September  5,  2001,  the  AOTC  submitted  a  revised  IT  Plan  and  an  assessment  of 
the  status  of  the  project  to  the  Legislature.       The  revised  IT  Plan  calls  for  the  completion 
of  the  IT  Project  by  June  30,  2005. 292  In  addition,  the  AOTC  states,  "it  is  anticipated  that 
the  IT  Project  will  be  completed  within  that  budget  ($75,000,000),  without  the  need  to 
seek  additional  bond  funding."293  The  Bureau  is  unable  to  credit  the  assertion  that  the 
project  will  be  completed  by  2005  or  that  it  will  be  finished  with  the  remaining  balance  of 
$34.4  million  in  unexpended  bond  funds.294 

286  "Review  of  Recommendations  by  the  Project  Advisory  Board  to  Build  a  Statewide,  Comprehensive 
Case  Management  System,"  Cyrus  F.  Gibson,  May  10,  2000. 

287  District  Court  Buy/Build  Comments,  February  28,  2000. 

288  See,  CJAM  Remarks,  February  14,  1996;  and,  compare,  Bureau  Interview,  October  4,  2001. 

289  Bureau  Interview,  October  4,  2001 . 

290  Letter  to  the  Bureau,  October  5,  2001. 

291  See,  September  5,  2001  Legislative  Filing. 

292  Id. 
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As  stated  earlier,  the  budget  and  planning  sub-committee  of  the  PAB  and  PAB 
itself  did  not  participate  in  the  preparation  of  the  Revised  IT  Plan,  nor  were  they  given  an 
opportunity  to  review,  comment  or  adopt  the  document.295  In  fact,  many  requests  from 
the  PAB  for  a  comprehensive  budget  and  plan  have  been  ignored.296  Given  this  fact,  the 
documentation  submitted  by  the  AOTC  has  not  been  subject  to  comprehensive  internal 
scrutiny  and  analysis.  In  order  to  analyze  the  budget  that  has  been  included  within  the 
revised  IT  Plan,  the  Bureau  has  referenced  a  previously  prepared  Budget  scenario 
document  that  provided  estimates  for  numerous  components  of  the  IT  Project  that  are 
included  within  the  AOTC  revised  IT  Plan  and  budget.297 

In  January  of  1999,  Deloitte  did  not  believe  that  the  IT  Project  could  be 
completed  for  the  $75  million  unless  certain  project  requirements  were  deleted.29 '  As  of 
December  of  1998,  $29  million  had  been  spent  on  the  AOTC  IT  Project.299  Deloitte 
estimated  that  the  remaining  effort  would  cost  $64  million.300  This  would  result  in  a 
project  deficit  of  $18  million.301  In  order  to  bring  the  project  within  the  $75  million, 
Deloitte  recommended  deleting  $19  million  dollars  worth  of  work  from  the  intended 
project  scope.       Included  within  the  items  deleted  were  an  estimated  $4.1  million  dollars 
for  interim  systems  development  and  a  contingency  budget  of  $5.8  million.303  However, 
the  interim  systems  development  continued  and  through  this  date,  the  AOTC  is  still 
attempting  to  fully  deploy  the  system.  Further,  the  BasCOT  development  component  of 
the  interim  Systems  was  completed  two  years  behind  schedule  with  significant  added 
costs  related  to  change  orders.304  The  delay  and  cost  was  related  to  the  inability  of  the 
AOTC  and  the  departments  of  the  Trial  Court  to  reach  agreement  on  the  full  and 
comprehensive  functionality  of  the  interim  systems.305  As  of  July  of  1999,  124 
programming  changes  initiated  by  the  District  Court  to  the  BasCOT  civil  small  claims 
program  were  being  considered.  Among  these  was  the  District  Court's  preference  for 
using  the  word  "and"  rather  than  the  symbol  "&"  in  the  program.       The  documents 
reviewed  by  the  Bureau  did  not  provide  an  assessment  of  the  costs  related  to  these 
changes.  By  way  of  comparison,  during  the  same  period,  Deloitte  estimated  that  4 
programming  changes  to  the  BasCOT  Land  Court  program  would  add  $10,488.00  to  the 
development  cost  of  the  program.      Therefore,  the  savings  that  would  have  been  realized 
by  deleting  continued  development  of  the  interim  systems  was  never  realized.  In  fact,  the 


295  Bureau  Interview,  October  4,  2001;  and,  Letter  to  the  Bureau,  October  5,  2001. 

296  Bureau  Interview,  October  4,  2001. 

297  Deloitte  Budget  Scenario  Document,  January  1999. 
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302  Deloitte  Budget  Scenario  Document,  January  1999. 
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304  As  of  July  of  1999,  there  were  124  programming  changes  initiated  by  the  District  Court  to  the  BasCOT 
civil  program  being  considered.  Among  these  was  the  District  Court's  preference  for  the  program  using  the 
word  "and"  in  certain  instances  rather  than  the  symbol  "&."  During  the  same  period,  Deloitte  estimated 
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cost  of  the  development  and  deployment  of  the  interim  systems  was  increased  due  to 
change  orders. 

Casting  further  doubt  on  the  AOTC's  budgetary  assertions  is  the  budget  scenario 
document  prepared  by  Deloitte  in  January  of  1999. 308  Deloitte  previously  anticipated 
$10.8  million  would  be  required  for  "Change  Management."309  Change  management 
would  include  all  costs  related  to  the  training  of  personnel  with  basic  office  automation 
and  MassCourts.       The  revised  IT  Plan  only  budgets  $1.1  million  for  this  critical 
component  and  does  not  include  a  summary  of  the  component  costs.31  ]   Similarly,  the 
Deloitte  budgetary  scenario  estimated  that  $600,000  would  be  needed  to  implement  a 
Disaster  Recovery  System.312  The  revised  IT  Plan  only  budgets  $350,000.313  Finally,  the 
Deloitte  budgetary  scenario  estimated  that  $5.8  million  would  be  budgeted  for 
contingencies  that  may  arise  during  the  life  of  the  project.314  As  mentioned  earlier,  the 
BasCOT  change  order  process  makes  clear  that  contingencies  must  be  factored.315  The 
revised  IT  Plan  does  not  budget  for  any  contingencies.316  Therefore,  when  cost  overruns 
and  change  orders  do  occur,  the  AOTC  will  be  forced  to  reduce  the  functionality  of  the 
system  as  provided  in  the  Deloitte  System  Scope  Document.  The  AOTC  states  that  the 
successful  vendor  for  MassCourts  "will  proposes  a  system  that  can  provide  all  the 
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features  and  functionality  encompassed  within  the  Scope  document. . ."  Because  the 
AOTC's  budget  leaves  no  room  for  error,  any  deviation  from  the  budget  will  result  in 
either  diminished  system  functionality  or  a  budget  deficit. 

This  budgetary  comparison,  when  viewed  in  light  of  the  AOTC's  history  with  the 
project  relative  to  timelines,  casts  serious  doubt  on  the  assertions  that  the  project  can  be 
completed  within  the  time  frame  specified  or  for  the  money  allotted.  As  mentioned 
above,  BasCOT  was  two  years  behind  schedule.  Similarly,  the  Business  Requirements 
confirmation  project  was  18  months  late  and  the  comprehensive  Network  Infrastructure 
Design  is  apparently318  not  competed  even  though  the  original  IT  Plan  indicated  that  it 
should  be  one  of  the  first  items  accomplished  relative  to  the  IT  Project. 

Lastly,  and  importantly,  the  revised  IT  Plan  fails  to  fully  address  the  ability  of 
MassCourts  to  interface  with  external  governmental  entities.  The  bond  bill,  as  amended, 
mandated  that: 

"such  comprehensive  statewide  integrated  court  information  system  be  capable  of 
readily  exchanging  data  with  existing  or  proposed  information  systems  in  law 
enforcement  agencies,  public  safety  agencies,  prosecutorial  agencies  and  social 


308  Deloitte  Budget  Scenario  Document,  January  1999. 
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310  Id. 
See,  September  5,  2001  Legislative  Filing. 
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313  See,  September  5,  2001  Legislative  Filing. 
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315  See,  Small  Claims  BasCOT  Change  Requests,  July  1999;  and,  Land  Court  BasCOT  Change  Estimate, 
August  3,  1999. 

3.6  See,  September  5,  2001  Legislative  Filing. 

3.7  Id.  at  3. 

318  Bureau  Interviews,  August  2,  2001,  and  October  4,  2001. 
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service  agencies  of  the  commonwealth  and  that  the  chief  justice  for  administration 
and  management  or  his  designee  shall  consult  with  the  secretary  of  public  safety 
ad  the  secretary  of  human  services,  or  their  designees,  in  designing  and 
implementing  such  a  system  (emphasis  added)."  St.  1996,  c.186,  §3. 

It  is  the  position  of  the  AOTC  that  these  external  entities  will  be  consulted 
regarding  their  ability  to  interface  with  the  system,  but  that  these  entities  will  not  take  a 
role  in  designing  or  implementing  the  system.319  This  position  is  in  direct  contravention 
with  the  express  words  of  the  statute.  The  failure  to  incorporate  this  important  element 
further  undermines  the  budgetary  and  planning  assessments  of  the  AOTC  as  revised  in 
September  of  200 1 . 

Therefore,  pending  a  detailed  analysis  and  comparison  by  the  PAB  or  another 
analytical  entity,  the  Bureau  cannot  confidently  find  that  this  IT  Project,  as  delineated  in 
the  System  Scope  document,  will  be  completed  for  the  $75  million  or  by  June  30, 

2005.320 

Recommendations : 

Develop  a  detailed  and  comprehensive  daily  project  management  plan  that  will 
allow  the  Project  Manager  to  monitor  the  progress  of  the  project  and  the 
performance  of  the  vendors; 

•  Establish  an  RFR  vendor  selection  process  that  represents  a  broad  cross-section  of 
the  AOTC  and  the  departments  of  the  Trial  Court; 

•  Establish  a  system  design  committee  that  will  report  to  the  Project  Manager  and 
will  have  authority  relative  to  the  design  of  a  developed  system  or  a  system  that  is 
adapted  from  an  existing  commercially  system; 

Use  Independent  Validation  and  Verification  (IVV)  consultants  on  complex 
components  of  the  IT  Project; 

Consult  with  the  agencies  referenced  at  St.  1996,  c.186,  §3  relative  to  the  design 
and  implementation  of  the  system; 

•  Consult  with  the  ITD  of  the  Executive  Office  of  Administration  and  Finance  on, 
at  least,  quarterly  basis  relative  to  the  status  of  the  project. 

•  The  position  of  Project  Director  should  be  made  a  full-time  judicial  position.  The 
judge  selected  for  this  enhanced  role  should  be  a  well-respected  veteran  of  the 
Trial  Court  with  management  skills,  political  acumen  and  technical  savvy  who 
can  work  effectively  with  the  Project  Manager. 


319  Bureau  Interviews,  August  9,  2001;  September  5,  2001  Legislative  Filing. 

120  Compare,  Deloitte  Budget  Scenario  Document,  January  1999;  and,  September  5,  2001  Legislative 


Filing. 
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Appendix  A: 
Suggested  IT  Project  Management  Practices 

The  Commonwealth  of  Massachusetts  lacks  a  uniform,  systemic,  approach  to  IT 
project  decision-making.321  The  AOTC  has  not  adopted  policies  or  decision-making 
guidance  documents  relative  to  its  IT  project.  Because  the  Commonwealth  of 
Massachusetts  also  lacks  formal  regulations,  policies  or  guidelines  that  are  applicable  to 
the  management  of  IT  projects,  the  Bureau  has  reviewed  a  number  of  governmental 
entities'  IT  regulations  and  guidelines.  Based  upon  that  review,  the  Bureau  has  relied 
upon  the  guidelines  and  rules  established  by  the  United  States  General  Accounting 
Office's  Accounting  and  Information  Management  Division. 

In  assessing  the  AOTC's  performance  relative  to  this  project,  the  Bureau  has 
adopted  criteria  and  practices  utilized  by  the  Federal  Government  to  assist  it  in  the 
evaluation  of  the  AOTC's  IT  decision-making  process.322 

Three  distinct  phases  of  a  project's  lifecycle  must  be  analyzed  and  incorporated 
by  an  agency:  project  selection,  project  control  and  project  evaluation. 

I.         Project  Phases 

Project  Selection 

IT  projects  require  a  screening  process  to  ensure  that  projects  selected  meet 
minimum  requirements  and  identify  how  the  project  should  be  reviewed.  The  costs, 
benefits  and  risks  of  the  individual  project,  whether  it  is  proposed,  under  development,  or 
operational,  should  be  assessed      and  compared  against  other  projects  to  gauge  how  well 
the  project  meets  the  mission  of  the  department. 

Finally,  a  senior  management  decision-making  body  should  be  charged  with 
making  the  final  decision  about  funding  the  project.  This  process  helps  ensure  that  the 
organization  selects  the  projects  that  comport  with  the  mission  and  helps  identify  the 
risks  and  proposed  benefits  before  a  significant  amount  of  project  funds  are  spent. 


321 

The  Information  Technology  Division  of  the  Executive  Office  of  Administration  and  Finance  has 

assumed  the  statutory  responsibility  of  coordinating  IT  investments  in  excess  of  $200,000.00.  This 

oversight,  however,  is  not  reduced  to  a  regulatory  scheme  nor  does  it  apply  to  certain  agencies,  such  as  the 

Judiciary. 

322  Assessing  Risks  and  Returns:  A  Guide  for  Evaluating  Federal  Agencies '  IT  Investment  Decision-making. 
(GAO/AIMD-10.1.13,  February  1997).  The  guide  is  based  upon  an  analysis  of  management  practices  of 
several  leading  private  and  public  sector  organizations,  and  a  comparison  of  the  management  practices  of 
federal  agencies  with  those  of  leading  organizations.  For  a  review  of  these  management  practices  see, 
Executive  Guide:  Improving  Mission  Performance  Through  Strategic  Information  Management  and 
Technology  (GAO/AIMD-94-1 15,  May  1994);  and,  Information  Technology  Investment:  Agencies  Can 
Improve  Performance,  Reduce  Costs,  and  Minimize  Risks  (GAO/AIMD-96-64,  Sept.  30,  1996). 

323  In  order  to  maintain  any  measure  of  objectivity,  the  project  should  be  assessed  and  evaluated  by  an 
outside  entity. 
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Project  Control 

Once  the  projects  are  selected,  they  must  be  consistently  controlled  and  managed. 
IT  Project  reviews,  in  which  the  progress  of  projects  are  compared  against  project  costs, 
schedule,  and  expected  mission  benefits,  are  conducted  at  key  milestones  in  the  project's 
lifecycle.  If  a  project  is  late,  over  cost,  or  not  meeting  performance  expectations,  senior 
executives  decide  whether  it  should  be  continued,  modified  or  cancelled,  and  actions  are 
quickly  taken  to  mitigate  the  effects  of  changes  in  risks  and  costs. 

Project  Evaluation 

The  last  step  in  the  project  lifecycle  requires  the  evaluation  of  actual  versus 
expected  results.  This  evaluation  should  focus  on  three  components.  One,  assessing  the 
projects  impact  on  mission  performance,  two,  identifying  changes  or  modifications  to  the 
project  that  may  be  needed,  three,  reviewing  the  management  process  based  upon  lessens 
that  were  learned. 

To  gauge  the  effectiveness  of  the  organization,  the  above  three-phase  process  is 
reviewed  by  focusing  on  two  major  levels  of  analysis:  process  and  data. 

II.        Project  Review 

Process 

The  goal  in  assessing  an  agency's  process  is  to  identify  to  what  extent  the 
organization  has  a  structure  in  place  for  managing  and  evaluating  its  IT  investments. 
There  should  be  clearly  articulated  and  documented  policies  that  establish  management 
processes  relative  to  the  selection,  control  and  evaluation  of  IT  investments.  The 
decision-making  roles,  responsibilities  and  authority  of  key  personnel  must  be  clearly 
defined  in  the  policy. 

Data 

IT  projects  must  rely  upon  accurate,  reliable,  and  up-to-date  data  on  project  costs, 
benefits  and  risks.  Documentation  of  management  decisions  is  essential  to  compile  a 
record  of  results.  Specifically,  all  projects  should  have  complete  and  accurate  project 
information  relative  to: 

Cost  and  benefit  data 

Risk  assessments 

Links  to  project  goals,  objectives  and  performance  measures 

Current  project  costs 

Implementation  plans 

Staffing  plans 

In  addition,  qualitative  and  quantitative  project  requirements  and  criteria  should 
be  established  to  assist  with  the  selection,  control  and  evaluation  of  projects  as  they  move 
through  their  respective  lifecycles. 
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III.      Suggested  Management  Best  Practices  for  IT  investments 

a.  Project  Selection  —  Process 

i.    Defined  policy  guideline  to  submit  and  screen  proposals  for 
consideration; 

ii.    Defined  business  case  justification  for  each  proposal; 

iii.    A  clear  delineation  and  explanation  of  roles,  responsibility  and 
authority  relative  to  the  selection  process; 

iv.    Defined  policy  providing  a  schedule  of  required  documentation  for 
IT  project  proposals,  which,  at  a  minimum  would  include; 

1 .  links  to  business  objectives  and  IT  strategic  or  project  plan; 

2.  identification  of  proposed  benefits,  both  qualitative  and 
quantitative  (cost/benefit  estimates,  alternatives  analyses, 
risk  assessment); 

v.    Defined  thresholds  to  for  cost/benefit  ratios,  return  on  investment 
calculations,  and  risk  assessments; 

vi.    Defined  validation  mechanism  for  IT  proposals  that  states  who  is 
responsible  for  the  validation  and  where  exceptions  are  permitted; 

vii.    Formal,  systematic  process  for  determining  priorities  and  making 
funding  decisions  that  clearly  establishes  who  has  the 
responsibility  and  authority  for  making  final  funding  decisions; 

viii.    Defined  and  established  IT  investment  review  group; 

ix.    Defined  process  to  monitor  approved  projects. 

b.  Project  Selection  —  Data 

i.    Process  to  ensure  that  all  of  the  required  data/information  that  was 
prepared  in  support  of  the  proposal  complied  with  the  prescribed 
process; 

ii.    Process  to  ensure  that  data/information  submitted  was  validated; 

iii.    Process  to  ensure  that  all  project  costs  have  been  identified; 

iv.    Process  to  ensure  that  all  benefits  and  risks  have  been  identified; 

v.    Process  to  ensure  that  project  schedule  information  was  reviewed 
and  validated; 
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vi.    Defined  outlined  and  strategies  to  handle  the  necessary 
acquisitions. 

- 

c.  Project  Control  -  Process 

i.    Defined,  documented  and  repeatable  process  for  monitoring  and 
reviewing  the  project  which,  at  a  minimum,  will  include; 

1 .  Project  status  review; 

2.  Estimated  vs.  actual  costs  to  date; 

3.  Estimated  schedule  vs.  actual  schedule; 

4.  Actual  outcomes  from  modular  testing  vs.  estimates; 

5.  Updates  on  risk  mitigation  efforts; 

6.  Technical  issues  that  have  arisen; 

7.  Contractor  performance  and  delivery; 

8.  Review  of  the  systems  methodology  process;  and 

9.  Changes  or  unexpected  issues  that  affect  project  progress 
and  outcomes; 

ii.    Defined  explanation  of  what  data  and  information  that  must  be 
regularly  submitted  for  management  evaluation; 

iii.    Defined  process  to  validate  and  verify  data  and  information 
submitted  for  review; 

iv.    Defined  group  responsible  for  control  management  with  specified; 

1 .  Rules  for  review  and  decision-making; 

2.  Defined  roles,  responsibilities  and  authority; 

3.  Defined  purpose  or  charter; 

v.    Defined  mechanisms  to  ensure  compliance  with  the  review 
process; 

vi.    Defined  membership  for  the  review  group  from  among  courts,  IT 
department,  financial  affairs  and  quality  assurance  or  outside 
assessment  group; 

vii.    Defined  procedures  to  ensure  the  accuracy  and  reliability  of  project 
information,  including; 

1.  Project-related  decisions; 

2.  Actions  to  be  taken  and  criteria  to  measure  improvement; 

3.  Project  outcomes; 

4.  Cost/benefit  analyses; 

5.  Business  case  information; 

viii.    Defined  process  regarding  the  maintenance  of  data; 
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ix.    Defined  process  to  evaluate  the  decision-making  process  and  to 
suggest  changes  based  upon  lessons  learned; 

d.  Project  Control  -  Data 

i.    Defined  measures  of  performance  to  track  costs,  schedules, 
benefits,  and  risks; 

ii.    Defined  process  to  track  actual  project  performance  against  the 
estimates  that  were  used  to  justify  the  project; 

iii.    Defined  process  to  analyze  gaps  between  estimates  and  actuals  and 
the  submission  of  explanatory  factors  for  positive  or  negative 
variances; 

iv.    Documentation  to  validate  and  verify  interim  cost,  schedule, 
benefit  and  risk  information  as  they  are  submitted; 

v.    Process  to  maintain  and  update  project  data; 

vi.    Defined  risk  mitigation  plans  to  address  deviations  in  schedule  and 
cost; 

vii.    Defined  process  to  measure  contractor  performance; 

e.  Project  Evaluation  -  Process 

i.    Defined,  documented  process  for  conducting  post-implementation 
reviews  (PIR)  if  IT  projects,  which  include; 

1 .  Purpose  of  the  PIR  process; 

2.  Delineation  of  roles,  responsibilities  and  authority  of 
people  involved  in  conducting  the  PIR  process; 

3.  Statement  of  how  conclusions  and  recommendations  are  to 
be  reviewed  and  communicated 

4.  Creation  of  a  standardized  methodology  for  conducting 
PIR; 

5.  Independent  validation  and  verification  of  PIR; 

ii.    Mechanism  to  change  the  decision-making  process  and  project 
management  as  a  result  of  the  PIR; 

iii.    Mechanism  to  assess  the  PIR  process. 

/  Project  Evaluation  -  Data 

i.    Process  to  collect  projected  versus  actual  cost,  benefit,  and  risk  as 
part  of  the  PIR,  including, 
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1 .  Cost,  benefit,  and  risk  information  that  was  used  for  initial 
project  justification; 

2.  Updates  to  costs,  benefit,  and  risk  information  that  were 
compiled  during  the  life  of  the  project; 

3.  Verification  and  validation  of  the  cost  data; 

ii.    Process  to  include  end-user  survey  and  comments  within  the  PIR; 

iii.    Process  to  include  technical  capability  assessment  as  part  of  the 
PIR; 

iv.    Process  to  include  actual  project  performance  versus  estimated 
performance  within  the  PER; 
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Appendix  B: 


Recommendations  for  AOTC  Project  Management 


To  improve  the  ability  of  the  AOTC  to  screen  and  select  component  parts  of  the  IT 
project,  the  AOTC  should  take  the  following  actions: 

Establish  an  RFR  and  acquisition  process  that  allows  vendors  to  propose  solutions 
for  Mass  Courts; 

Establish  defined  policy  guidelines  to  submit  and  screen  individual  project 
proposals  for  consideration; 

•  Establish  defined  business  case  justifications  for  each  individual  project  proposal; 

•  Establish  defined  policies  providing  a  schedule  of  required  documentation  for 
individual  project  proposals,  which,  at  a  minimum  would  include; 

1 .  links  to  business  objectives  and  the  Revised  IT  Plan; 

2.  identification  of  proposed  benefits,  both  qualitative  and 
quantitative  (cost/benefit  estimates,  alternatives  analyses, 
risk  assessment); 

•  Develop  defined  thresholds  for  cost/benefit  ratios,  return  on  investment 
calculations,  and  risk  assessments; 

•  Develop  a  defined  validation  mechanism  for  individual  project  proposals  that 
states  who  is  responsible  for  the  validation  and  where  exceptions  are  permitted; 

•  Use  Independent  Validation  and  Verification  (IW)  consultants  on  complex 
components  of  the  IT  Project; 

•  Develop  a  formal,  systematic  process  for  determining  priorities  and  making 
funding  decisions  that  clearly  establishes  who  has  the  responsibility  and  authority 
for  making  final  funding  decisions; 

•  Establish  a  defined  and  established  IT  investment  review  group; 

•  Develop  a  defined  process  to  monitor  approved  individual  project  proposals; 

•  Develop  a  defined  process  to  ensure  that  all  of  the  required  data/information  that 
was  prepared  in  support  of  the  individual  project  proposal  complied  with  the 
prescribed  process; 

•  Develop  a  defined  process  to  ensure  that  any  data/information  submitted  in 
support  of  an  individual  project  proposal  was  validated; 
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Develop  a  defined  process  to  ensure  that  all  individual  project  proposal  costs  have 
been  identified; 

Develop  a  defined  process  to  ensure  that  all  individual  project  proposal  benefits 
and  risks  have  been  identified; 

Develop  a  defined  process  to  ensure  that  individual  project  proposal  schedule 
information  was  reviewed  and  validated; 

Establish  defined  and  outlined  strategies  to  handle  the  necessary  acquisitions 
related  to  individual  project  proposals; 

To  improve  the  ability  of  the  AOTC  to  control  the  IT  project,  the  AOTC  should  take  the 
following  actions: 

•  Establish  a  defined,  documented  and  repeatable  process  for  monitoring  and 
reviewing  the  project,  which,  at  a  minimum,  will  include; 

1.  Project  status  review; 

2.  Estimated  versus  actual  costs  to  date; 

3.  Estimated  schedule  versus  actual  schedule; 

4.  Actual  outcomes  from  modular  testing  versus  estimates; 

5.  Updates  on  risk  mitigation  efforts; 

6.  Technical  issues  that  have  arisen; 

7.  Contractor  performance  and  delivery; 

8.  Review  of  the  systems  methodology  process;  and 

9.  Changes  or  unexpected  issues  that  affect  project  progress 
and  outcomes; 

•  Establish  a  defined  explanation  of  what  data  and  information  must  be  regularly 
submitted  for  management  evaluation; 

•  Establish  a  defined  process  to  validate  and  verify  data  and  information  submitted 
for  review; 

•  Designate  a  defined  committee  responsible  for  control  management  with 
specified; 

1 .  Rules  for  review  and  decision-making; 

2.  Defined  roles,  responsibilities  and  authority; 

3.  Defined  purpose  or  charter; 

Establish  a  defined  mechanisms  to  ensure  compliance  with  the  review  process; 

Establish  defined  membership  for  the  review  group  from  among  courts,  IT 
department,  financial  affairs  and  quality  assurance  or  outside  assessment  group; 

•  Develop  defined  procedures  to  ensure  the  accuracy  and  reliability  of  project 
information,  including; 
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1.  Project-related  decisions; 

2.  Actions  to  be  taken  and  criteria  to  measure  improvement; 

3.  Project  outcomes; 

4.  Cost/benefit  analyses; 

5.  Business  case  information; 

Develop  a  defined  process  regarding  the  maintenance  of  data; 

Establish  a  defined  process  to  evaluate  the  decision-making  process  and  to 
suggest  changes  based  upon  lessons  learned; 

Establish  a  defined  measures  of  performance  to  track  costs,  schedules,  benefits, 
and  risks; 

Establish  a  defined  process  to  track  actual  project  performance  against  the 
estimates  that  were  used  to  justify  the  project; 

Establish  a  defined  process  to  analyze  gaps  between  estimates  and  actuals  and  the 
submission  of  explanatory  factors  for  positive  or  negative  variances; 

Require  documentation  to  validate  and  verify  interim  cost,  schedule,  benefit  and 
risk  information  as  they  are  submitted; 

Establish  a  process  to  maintain  and  update  project  data; 

Develop  a  defined  risk  mitigation  plans  to  address  deviations  in  schedule  and 
cost; 

Establish  a  defined  process  to  measure  contractor/vendor  performance; 
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Glossary324 

Benefit:  term  used  to  indicate  an  advantage,  profit,  or  gain  attained  by  an  individual  or 
organization. 

Best  Practice:  the  processes,  practices,  or  systems  identified  in  public  and  private 
organizations  that  performed  exceptionally  well  and  are  widely  recognized  as  improving 
an  organization's  performance  and  efficiency  in  specific  areas.  Successfully  identifying 
and  applying  best  practices  can  reduce  business  expenses  and  improve  organizational 
efficiency. 

Business  Case:  a  structured  proposal  for  business  improvement  that  functions  as  a 
decision  package  for  organizational  decision-makers.  A  business  case  includes  an 
analysis  of  business  process  performance  and  associated  needs  or  problems,  proposed 
alternative  solutions,  assumptions,  constraints,  and  a  risk-adjusted  cost-benefit  analysis. 

Cost/benefit  analysis:  a  technique  used  to  compare  the  various  costs  associated  with  an 
investment  with  the  benefits  that  it  proposes  to  return.  Both  tangible  and  intangible 
factors  should  be  addressed  and  accounted  for. 

Information  Resource  Management  (IRM):  the  process  of  managing  information 
resources  to  accomplish  agency  missions.  This  term  encompasses  information  itself,  as 
well  as  related  resources,  such  as  personnel,  equipment,  funds,  and  information 
technology. 

Information  Technology  (IT):  the  hardware  and  software  operated  by  an  organization  to 
accomplish  a  state  function,  regardless  of  the  technology  involved  (e.g.,  computers, 
telecommunications,  etc.). 

Performance  Gap:  the  gap  between  what  customers  and  stakeholders  expect  and  what 
each  process  and  related  sub  processes  produces  in  terms  of  quality,  quantity,  time,  and 
cost  of  services  and  products. 

Performance  Measurement:  the  process  of  developing  measurable  indicators  that  can 
be  systematically  tracked  to  assess  progress  made  in  achieving  predetermined  goals  and 
using  such  indicators  to  assess  progress  in  achieving  these  goals. 

Post-implementation  Review  (PIR):  an  evaluation  tool  that  compares  the  conditions 
prior  to  the  implementation  of  a  project  (as  identified  in  the  business  case)  with  the  actual 
results  achieved  by  the  project. 

Return  on  Investment  (ROI):  a  figure  of  merit  used  to  help  make  capital  investment 
decisions.  ROI  is  calculated  by  considering  the  annual  benefit  divided  by  the  investment 
amount. 

Risk  Analysis:  a  technique  to  identify  and  assess  factors  that  may  jeopardize  the  success 
of  a  project  or  achievement  of  a  goal.  This  technique  also  helps  define  preventive 


324  Adopted  from  the  United  States  General  Accounting  Office. 
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measures  to  reduce  the  probability  of  these  factors  from  occurring  and  identify 
countermeasures  to  successfully  deal  with  these  constraints  when  they  develop. 

Strategic  Planning:  a  systematic  method  used  by  an  organization  to  anticipate  and  adapt 
to  expected  changes.  The  information  resource  management  portion  of  strategic  planning 
sets  broad  direction  and  goals  for  managing  information  and  supporting  delivery  of 
services  to  customers  and  the  public  and  identifies  major  information  resource 
management  activities  to  be  undertaken  to  accomplish  the  desired  agency  mission  and 
goals. 
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Appendix  C 


AOTC  Response  and  Comment 
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ADMINISTRATIVE  OFFICE  OF  THE  TRIAL  COURT 
Two  Center  Plaza 

Boston.  Massachusetts  02108 

BARBARA  A.  DORTCH-OKARA  Tel:  (617)  742-8575 

Chief  Justice  for  Fax:  (617)  742-0968 

Administration  and  Management 


November  16,  2001 


Honorable  James  H.  Fagan 

Chairman,  Committee  on  Post  Audit  and  Oversight 

State  House,  Room  146 

Boston,  MA  02780 

Dear  Chairman  Fagan: 

I  very  much  appreciate  the  opportunity  to  offer  these  comments  on  the  draft  of  the 
House  Post  Audit  and  Oversight  Bureau's  Preliminary  Report  on  the  Massachusetts  Trial 
Court's  administration  of  the  "Judiciary  Information  Technology  Loan  Act  of  1995."  I 
understand  that  these  comments  wili  be  incorporated  into  the  final  report. 

First,  I  would  like  to  commend  you  and  your  staff  for  the  professional  and  courteous 
manner  in  which  this  inquiry  has  been  handled.  As  I  stated  in  my  letter  to  you  of  July  17, 
2001,  "[w]e  share  a  common  goal  of  addressing  any  deficiencies  that  may  exist  and 
ensuring  that  this  project  is  completed  on  time  and  within  budget."  To  that  end,  the  draft 
report  that  I  have  reviewed  contains  many  very  helpful  suggestions  and  some  much 
deserved  criticism.  Before  turning  to  the  specific  criticisms  however,  it  is  important  to  point 
out  the  considerable  accomplishments  of  the  IT  Project  to  date. 

BasCOT  has  in  fact  automated  100%  of  the  Boston  Municipal  Court's  criminal  case 
load,  100%  of  the  Land  Court's  case  load,  and  nearly  100%  of  the  Probate  and  Family 
Court's  case  load.  The  Bond  funds  have  also  provided  the  Superior  Court  with  100% 
automation,  ForeCourt  Vision.  As  a  result  of  this  infiltration  of  automation,  numerous  court 
employees  have  received  technological  tools  ,  software,  hardware  and  training,  to  assist 
them  in  conducting  the  court's  business.  Also,  the  employees  have  become  versed  in 
modern  day  tools  which  will  assist  them  in  transitioning  to  MassCourts. 

When  I  first  assumed  the  duties  of  Chief  Justice  for  Administration  and  Management 
I  quickly  became  aware  of  the  need  to  improve  the  management  structure  of  the  IT  Project. 
I  set  about  the  process  of  finding  and  hiring  a  full  time,  experienced  Project  Manager. 
When  the  recruitment  and  interview  process  failed  to  produce  an  appropriate  candidate, 
I  created  the  position  of  Project  Director  and  enlisted  a  sitting  justice  of  the  Trial  Court  to 


serve  in  that  capacity  to  provide  the  necessary  leadership.  Efforts  continued,  however,  to 
find  a  Project  Manager,  and  that  position  was  filled  in  May,  2000.  I  also  understood  the 
need  for  a  more  streamlined  and  effective  committee  structure  to  advise  the  project  on 
policy  issues.  To  that  end,  I  created  the  Project  Advisory  Board,  the  larger  user  focus 
group  called  CMAT  and  the  Project  Executive  Committee.  It  was  with  my  full  support  that 
the  Project  Advisory  Board  established  task  forces,  design  groups  and  committees, 
including  the  Budget  and  Planning  Committee.  These  changes  resulted  in  marked 
improvement  in  the  progress  of  the  IT  Project. 

Notwithstanding  the  improvements  that  have  been  made  within  the  last  three  years, 
I  have  not  been  satisfied  that  the  administrative  oversight  of  the  IT  Project  was  adequate. 
In  recent  months  changes  have  been  made  at  the  highest  levels  to  address  some  of  those 
shortcomings.  It  has  also  become  clear  that  the  IT  Project  needs  new  leadership  and  a 
more  authoritative  organizational  structure.  It  is  equally  clear  that  detailed  and  specific 
plans  must  be  in  place  to  adequately  measure  the  project's  progress  toward  ultimate 
completion.  Reports  must  be  produced  to  provide  the  necessary  oversight  to  the  project. 
According  to  the  plan  that  I  outlined  in  my  previous  letter  to  you,  I  commissioned  an 
independent  programmatic  review  of  the  entire  project  by  a  multi-disciplinary  committee  of 
well-respected  judges,  representatives  of  the  Executive  Branch  and  members  of  the  legal 
community  chaired  by  Justice  James  F.  McHugh  of  the  Appeals  Court.  This 
comprehensive  review  has  resulted  in  a  set  of  proposals  that  I  have  accepted  and  that  are 
currently  being  implemented.  Many,  if  not  most,  of  the  recommendations  made  by  the  IT 
Review  Committee  mirror  those  contained  in  your  draft  report. 

I  would  like  to  provide  you  with  the  most  recent  and  up-to-date  information 
concerning  the  IT  Project.  Based  upon  the  recommendations  of  the  IT  Review  Committee 
and  consistent  with  the  Bureau's  report,  a  new,  strong  and  clear  management  structure  for 
the  IT  Project  has  been  developed.  This  new  management  structure  will  provide  the  clear 
demarcation  of  authority  and  control  that  this  project  needs.  Through  this  new 
management  structure,  the  individual  departments  of  the  Trial  Court  will  understand  that 
the  decision  making  authority  for  all  aspects  of  the  IT  Project  rises  in  a  straight  line  to  me. 
The  IT  Project  Office  will  operate  as  an  entity  distinct  from  the  AOTC's  IT  Department,  with 
reporting  responsibility  directly  to  me.  A  new  position  of  Project  Executive  has  been 
created.  This  position  will  be  filled  by  a  leader  in  the  Judicial  Branch  who  will  devote  full 
time  to  the  project.  A  respected  justice  or  administrator  is  being  sought  for  this  important 
position.  He  or  she  will  be  accountable  directly  to  me  and  will  have  full  and  final  authority 
to  make  decisions  concerning  the  project,  subject  only  to  my  veto.  The  Project  Advisory 
Board  will  serve  as  a  resource  to  the  Project  Executive  and  will  continue  to  advise  on 
matters  of  policy.  All  other  components  of  the  project  will  report  to  the  Project  Executive 
through  the  Project  Manager. 

A  new  Project  Manager  has  been  hired.  Susan  A.  Laniewski,  Esq.  began  work  on 
Tuesday,  November  1 3, 2001 .  A  copy  of  her  resume  is  enclosed.  As  you  will  see,  she  has 
had  extensive  experience  in  managing  information  technology  projects  not  only  in  the 
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public  sector,  but  also  projects  concerning  the  automation  of  court  systems.  The  wealth  of 
experience  that  Ms.  Laniewski  brings  with  her  from  her  work  on  the  automation  of  court 
systems,  particularly  in  her  work  as  Director  of  Technology  for  the  National  Center  for  State 
Courts,  will  be  a  tremendous  benefit  to  our  project  at  this  critical  stage.  The  Project 
Manager  will  report  to  the  Project  Executive  and  will  be  responsible  for  all  aspects  of  the 
day-to-day  management  of  the  project,  including  budget  and  resources.  There  will  also 
be  four  "Project  Expediters",  one  from  each  of  the  four  largest  Trial  Court  departments  who 
will  work  with  and  be  responsible  to  the  Project  Manager  for  application  design,  rollout, 
change  management,  and  a  variety  of  other  management  and  administrative  issues  that 
arise  with  this  project.  In  addition  to  implementing  this  new  management  structure,  the 
following  action  has  been  or  will  be  taken: 

I  have  directed  the  new  Project  Manager  to  develop,  within  the  next 
thirty  days,  the  necessary  detailed  and  comprehensive  management 
plan  for  the  project.  This  plan  will  amplify  the  IT  Plan  filed  with  the 
various  Legislative  committees  in  September  of  this  year.  Thereafter, 
monthly  reports  will  be  produced  based  upon  that  management  plan 
indicating  progress  to  date  and  identifying  any  deviations. 

Comprehensive  policies  and  procedures  will  be  developed  to  guide 
the  organization  through  the  remainder  of  the  project.  While  all  work 
on  MassCourts  will  not  be  suspended,  we  will  not  move  forward  with 
the  RFR  process  until  the  new  structure,  policies  and  procedures  are 
developed  and  in  place. 

The  Project  Manager  will  henceforth  produce  schedules  of  anticipated 
expenditures  of  funds  that  will  include  a  detailed  itemization  of  the 
Bond  funds  that  will  be  expended  and  what  will  be  purchased  with  the 
funds.  The  AOTC  Audit  Division  has  been  directed  to  produce  regular 
reports  concerning  all  expenditures  of  funds  (Bond  and  general 
appropriation)  that  are  used  for  the  IT  Project.  These  reports  will  be 
used  to  ensure  that  all  expenditures  are  made  in  accordance  with  the 
IT  Plan,  as  approved  by  the  Chief  Justice  for  Administration  and 
Management  and  filed  with  the  Legislature.  The  Administrative  Office 
of  the  Trial  Court  has  defined  "administration  and  implementation" 
costs  as  including  those  costs  associated  with  the  management  or 
supervision  of  day-to-day  operations  of  the  Project  (so  called 
"overhead"costs).  These  are  the  expenses  related  to  the 
management  of  the  Project  and  its  implementation.  This  definition  is 
consistent  with  that  used  by  other  public  governmental  agencies. 

Clear  and  explicit  instructions  have  been  given  to  the  Project  Manager 
to  consult  with  the  agencies  referred  to  in  St.  1 996,  C.  1 86  §  3  relative 
to  the  design  and  implementation  of  the  Trial  Court's  Information 
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Technology  Project.  Further,  the  Project  Manager  will  consult  with  the 
ITD  on  at  least  a  quarterly  basis  regarding  the  progress  of  this  project. 

As  the  RFR  process  moves  forward  to  seek  a  vendor  for  the 
comprehensive  case  management  system,  a  selection  committee  will 
be  formed  that  appropriately  represents  a  cross  section  of  the  AOTC 
and  its  departments,  divisions  and  commissions. 

The  Acting  Chief  of  Staff  has  compiled  a  comprehensive  list  of  all 
statutory  directives  affecting  the  AOTC,  its  departments,  divisions  and 
commissions.  I  have  instructed  her  to  review  the  list  at  least  semi- 
annually to  ensure  that  there  is  full  compliance.  I  have  also  put  in 
place  a  procedure  whereby  the  Trial  Court's  General  Counsel  is 
responsible  for  providing  me  with  an  analysis  of  any  and  all  legislation 
that  directly  or  indirectly  affects  the  Administrative  Office  of  the  Trial 
Court  or  any  of  its  departments,  divisions  or  commissions.  The 
analyses  will  be  circulated  to  all  affected  individuals  and  offices. 

I  would  note  also  that  as  Chief  Justice  for  Administration  and  Management,  I  have 
acted  upon  each  and  every  recommendation  presented  to  me  by  the  Project  Advisory 
Board.  I  have  accepted  most  recommendations,  at  times  I  have  requested  more 
information,  some  recommendations  have  not  been  accepted.  To  the  extent  that  the 
Bureau's  draft  report  contains  statements  that  the  "AOTC"  failed  to  follow  the 
recommendations  of  project  advisory  and  oversight  committees,  I  would  appreciate 
clarification. 

For  the  past  two  years  the  Administrative  Office  of  the  Trial  Court  has  made 
comprehensive  presentations  to  the  Legislature  to  explain  the  progress  that  has  been  made 
in  the  development  and  implementation  of  the  comprehensive  integrated  computer  system. 
Repeatedly,  we  have  emphasized  the  need  for  adequate  personnel  and  resources  in  our 
operational  budget  to  support  and  sustain  this  project.  Many  of  the  deficiencies  that  have 
been  cited  in  the  draft  report  can  be  attributed  to  the  lack  of  adequate  staffing.  I  know  this 
point  has  been  made  to  the  Bureau  through  numerous  individuals  who  have  provided 
information.  I  hope  that  you  will  join  with  me  in  supporting  our  continuing  requests  for 
adequate  resources  to  support  this  important  effort. 

Finally,  while  I  understand  that  the  Bureau  has  doubt  about  the  AOTC's  ability  to 
complete  this  project  within  the  $75  million  dollar  Bond  appropriation.  It  is  a  doubt  that  I  do 
not  share.  Nor  is  it  shared  by  the  IT  Review  Committee.  Further,  it  is  contrary  to  the  strong 
and  explicitly  expressed  views  of  the  former  Project  Manager,  the  Chairman  of  the  Project 
Advisory  Board  and  a  representative  of  Deloitte  Consulting,  all  of  whom  spoke  with  the 
Bureau  and  stated  their  individual  opinions  that  the  project  could  be  completed  within  that 
existing  appropriation. 

I  have  attached  a  set  of  comments  and  suggested  corrections  to  wording  used  in  the 
draft  report  that  I  would  commend  to  your  attention  and  request  that  these  changes  be 
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made.  Again,  I  appreciate  the  opportunity  to  make  these  comments  and  I  look  forward  to 
receiving  the  Bureau's  final  report. 


Very  truly  yours, 


Barbara  A.  Dortch-Okara 

Chief  Justice  for  Administration  and 

Management 


Enclosures 
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PROPOSED  CORRECTIONS  TO  THE 

HOUSE  POST  AUDIT  AND  OVERSIGHT  BUREAU'S 

PRELIMINARY  REPORT  ON  THE  MASSACHUSETTS 

TRIAL  COURTS  ADMINISTRATION  OF  THE 

"JUDICIARY  INFORMATION  TECHNOLOGY  LOAN  ACT  OF  1995' 


Page  2  and  Page  10 

The  final  bullet  (#6)  is  duplicative  of  the  third  bullet.  The  relevant  statutes  impose 
no  separate  obligation  on  the  Administrative  Office  of  the  Trial  Court  to  inform  the 
Legislature  of  deviations  from  the  IT  Plan  as  filed,  except  as  to  require  the  annual  filing  of 
an  accounting  and  assessment.  That  point  is  adequately  and  correctly  noted  in  the  third 
bullet.  There  is,  therefore,  no  need  to  include  the  last  bullet. 


Page  7 

The  third  full  paragraph  indicates  that  in  response  to  the  RFI,  vendors  made 
submissions  indicating  the  expected  cost  of  the  IT  Project.  That  is  true.  The  responses 
received  indicated  that  the  total  estimated  price  would  be  $72  million  dollars,  and  that  is  the 
appropriation  the  AOTC  sought  from  the  Legislature.  The  figure  of  $82  to  $98  million  is  not 
accurate. 
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In  the  first  sentence  of  the  first  full  paragraph,  note  is  made  of  the  AOTC's  failing  to 
notify  the  Legislature  of  the  progress  made  in  implementing  the  IT  Plan.  In  spite  of  the 
obvious  oversights  in  meeting  the  filing  requirements,  the  AOTC  has  never  attempted  to 
shield  the  Project  from  review  by  the  Legislature.  On  the  contrary,  the  AOTC  proudly 
displayed  the  progress  of  the  interim  systems  and  the  model  EACC  project  to  the 
Legislature  at  its  IT  Day  at  the  State  House  in  the  Spring  of  2000.  In  the  Spring  of  2001  it 
held  a  similar  but  smaller  scale  demonstration. 

Later  in  that  same  paragraph  the  assertion  is  made  that  "it  is  apparent  that  no  efforts 
were  made  to  establish  a  reporting  system  or  to  insure  that  the  project  proceeded 
according  to  the  plan  that  was  filed  with  the  Legislature."  The  cited  "Assistant  AOTC  IT 
Project  Manager  Notes,  July  16,  2001"  does  not  support  that  assertion.  Further,  it  is 
important  to  note  that  the  Chief  Justice  for  Administration  and  Management  has 
subsequently  taken  steps  to  address  this  issue.  The  Audit  Division  of  the  AOTC  has  been 
directed  to  produce  regular  financial  reports  detailing  the  amount  of  Bond  funds 
encumbered  and  specifying  the  purpose  of  each  such  expenditure,  by  category  -  including 
"administration  and  implementation"  expenditures.  The  General  Counsel  for  the  AOTC  was 
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directed  to  ensure  that  all  of  the  statutory  obligations  imposed  by  St.  1995,  Ch.  263,  as 
amended  by  St.  1995,  Ch.  186  are  met  in  a  timely  fashion  and  to  make  the  Chief  Justice 
for  Administration  and  Management  aware  immediately  of  any  deficiencies.  The  Project 
Manager  has  been  directed  to  produce  monthly  progress  reports  regarding  the  IT  Project. 

The  first  Finding,  as  appearing  on  page  1 2,  indicates  that  the  AOTC  did  not  comply 
with  St.  1992,  c.  133,  section  37.  That  statute  imposes  no  additional  obligation  on  the 
AOTC  beyond  those  obligations  contained  in  the  Bond  bills.  The  statute  was  adopted  to 
ensure  that  Trial  Court  Departments  would  no  longer  independently  undertake  informal 
technology  projects  that  were  not  in  conformance  with  the  IT  Plan.  It  is  important  to  note 
that  all  subsequent  expenditures  have  been  subject  to  the  approval  of  the  Chief 
Administrative  Justice  (now  CJAM),  and  all  IT  projects  that  have  been  developed  since 
have  been  part  the  AOTC's  overall  plan  for  computerization  of  the  Trial  Court. 
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In  the  second  full  paragraph,  the  "Assistant  AOTC  IT  Project  Manager  Notes,  July 
1 6,  2001 "  is  cited  as  support  for  the  proposition  that  "the  AOTC,  prior  to  July  2001  did  not 
define  what  project  expenses  would  be  categorized  as  being  related  to  administration  and 
implementation."  The  document  does  not  support  that  assertion. 

Because  the  terms  "administration  and  implementation"  are  not  defined  in  the  Bond 
bill,  the  AOTC  has  adopted  and  used  a  definition  and  a  working  mechanism  to  identify 
costs  related  to  "administration  and  implementation"  of  the  project.  Regular  financial 
reports  are  to  be  generated  by  the  AOTC  Audit  Division  detailing  those  expenditures. 
Those  expenditures  include  expenditures  for  salaries  of  personnel  responsible  for 
processing  the  fiscal  documents  and  keeping  the  fiscal  records  of  the  project;  legal  services 
required  to  evaluate  contracts  and  plans;  conference  and  travel  expenses  related  to 
administration  of  the  project;  overhead  expenses  related  to  training  including  space  rental, 
food  services,  equipment  and  software;  overhead  expenses  related  to  renovation  of  space 
to  house  project  staff  and  contractors  dedicated  to  the  project,  including  supplies,  tools, 
furniture  and  equipment;  and  overhead  expenses  of  staff  administering  the  project  including 
office  supplies,  postage  and  telephone  service. 
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In  the  second  full  paragraph  the  statement  is  made  that  "more  than  $40  million  has 
been  expended  without  any  apparent  plan."  It  would  be  more  accurate  to  state  that  there 
has  not  been  an  "updated  written  plan."  A  significant  and  identifiable  amount  of  progress 
has  been  made  within  internal  plans  developed  by  the  AOTC  in  coordination  with  the  Trial 
Court  Departments. 
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In  the  first  full  paragraph  the  Small  Claims  BasCOT  Change  requests,  June  of  1 999 
are  cited  as  support  for  the  proposition  that  "[t]he  lack  of  a  shared  vision  for  the  project  has 
resulted  in  project  delay  and  increased  cost."  The  document  does  not  provide  adequate 
support  for  the  proposition  asserted. 

In  the  second  full  paragraph  it  is  asserted  that  the  MIT  consultant  was  hired  to 
"provide  an  assessment  of  the  project. . ."  That  is  not  an  accurate  or  correct  statement.  The 
consultant  was  hired  to  provide  an  assessment  of  the  process  by  which  the 
recommendation  to  "build"  rather  than  to  "buy"  the  system  was  made. 
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The  first  full  paragraph  contains  the  statement  that  the  Project  Manager  position  was 
"neither  created  nor  filled  until  the  Spring  of  2000."  The  second  full  paragraph  states, 
"[notwithstanding  the  awareness  that  a  separate  Project  Manager  should  be  hired,  a 
Project  Manager  was  not  hired  until  2000."  These  statements  are  not  accurate.  The 
position  of  Project  Manager  was  created  and  posted  a  full  year  earlier.  When  the 
recruitment  and  interview  process  failed  to  produce  a  suitable  candidate,  the  Chief  Justice 
for  Administration  and  Management  created  the  position  of  Project  Director  and  named 
Judge  Anthony  Nesi  to  that  position  to  provide  the  necessary  leadership  in  moving  the 
project  forward  until  a  full  time  Project  Manager  could  be  hired. 


Page  26 

The  last  sentence  of  the  paragraph  that  begins  the  page  is  misleading.  The  CJAM 
met  on  a  weekly  basis  with  the  Chief  Information  Officer  and  the  Executive  Director  in 
regular  supervisory  sessions.  These  meetings  were  designed  to  provide  supervision  to  the 
CIO  and  were  not  for  the  purpose  of  making  decisions  regarding  the  IT  Project. 
Accordingly,  the  IT  Project  Manager  was  not  regularly  invited  to  attend  these  supervisory 
sessions.  At  a  minimum,  that  sentence  should  be  changed  to  read  that  the  Project 
Manager  believed  that  project  decisions  were  being  made  at  those  meetings. 

The  first  full  paragraph  is  also  misleading.  The  description  of  the  state  of  affairs  that 
confronted  the  Project  Manager  in  the  spring  of  2000  when  she  assumed  her  duties  is 
accurate.  However,  the  assertion  that  the  hiring  of  the  Project  Manager  "did  not  result  in 
the  establishment  of  systemic  processes  to  guide  the  management  and  the  decision- 
making process  of  the  IT  Project"  belies  the  considerable  improvement  that  the  Project 
Manager  did  put  in  place  before  her  resignation. 
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Footnote  142  cites  a  "contract  renewal  letter,  July  23,  1999."  In  fact,  the  document 
cited  is  nothing  more  than  a  request  to  the  Legal  Department  to  draft  new  language  for  a 
contract  extension.  No  new  language  was  added  to  the  contract.  This  document  does  not 
support  the  proposition  for  which  it  is  cited. 
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It  is  noted  in  the  first  sentence  of  the  first  paragraph  that  Deloitte  presented  a  project 
oversight  structure  to  the  AOTC  in  October  of  1 998.  It  should  be  noted  that  Chief  Justice 
Dortch-Okara  began  to  implement  that  structure  within  60  days  of  assuming  the  position 
of  Chief  Justice  for  Administration  and  Management. 


Page  31 

The  second  full  paragraph  sets  forth  the  Bureau's  concern  regarding  whether  the 
project  can  be  completed  within  the  funds  already  appropriated.  The  documents  cited  in 
support  of  the  Bureau's  concern  are  taken  out  of  context.  Deloitte  has  consistently 
maintained  that  the  comprehensive,  integrated  information  technology  system 
contemplated  by  the  Bond  legislation  can  be  fully  developed  and  implemented  with  the 
funds  that  have  already  been  appropriated.  The  documents  cited  were  produced  as  part 
of  planning  exercise  designed  to  define  the  full  scope  of  the  contemplated  project. 


Page  33 

Footnote  21 2-21 6  cite  a  document  entitled  "Potential  Drawback  to  BasCOT  Rollout." 
That  document  is  an  unsigned,  typewritten  note.  It  has  been  taken  out  of  context  and  used 
to  support  a  proposition  that  it  was  not  intended  to  support. 
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The  third  sentence  of  the  first  paragraph  should  read,  "[h]e  stated  that  he  was 
unaware  of  any  comprehensive  cost  analysis  or  budget  estimates  that  had  been  prepared 
by  the  AOTC-IT." 

Footnote  228  states  that,  "It  appears  that  a  Project  Manager  has  not  yet  been  hired." 
That  is  no  longer  accurate.  Susan  Laniewski  began  work  on  November  13,  2001. 
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The  third  full  paragraph  makes  the  point  that  the  new  Project  Manager  will  spend  a 
significant  amount  of  time  developing  plans,  procedures  and  policies.  One  of  the  reasons 
that  Susan  Laniewski  was  hired  was  because  she  brings  with  her  the  experience  in 
managing  large,  public  court  information  technology  projects  that  will  enable  her  to  have 
those  policies  and  plans  in  place  within  30  days. 


Page  38 

The  second  paragraph  asserts  that  the  appropriate  technical  and  application 
architecture,  the  appropriate  testing  mechanism  and  process,  ultimate  ownership  of  the 
software,  and  questions  of  damages  and  limitations  on  liability  were  focal  points  of 
contention  during  the  life  of  the  integrator  contract.  That  assertion  is  wrong  and  the 
document  cited  does  not  support  the  proposition.  In  fact,  the  document  cited  is  nothing 
more  than  a  request  to  the  legal  department  to  draft  new  language  for  a  contract  extension. 
No  new  language  was  added  to  the  contract. 


Page  39 

The  first  full  paragraph  addresses  the  likely  need  for  a  new  vendor  to  "validate"  not 
"verify"  the  business  requirements  and  scope  documents  prepared  by  Deloitte.  While  there 
is  no  disagreement  that  some  validation  will  be  necessary,  Deloitte  was  clear  and 
unambiguous  in  informing  the  Bureau  that  this  process  would  not  be  lengthy  or  expensive. 


Page  40 

In  the  final  paragraph  a  statement  by  Chief  Justice  Dortch-Okara  is  quoted.  The 
statement  was  made  in  explaining  the  decision  to  "build"  rather  than  to  "buy"  the  new 
system.  Footnote  270  intimates  that  the  Chief  Justice  was  wrong  in  her  assertion  that 
Deloitte  was  willing  to  assume  the  technical  risk  that  the  system  they  designed  would  not 
work  as  evidenced  by  the  breakdown  in  the  negotiations  between  the  AOTC  and  Deloitte. 
In  fact,  the  risk  that  Deloitte  was  not  willing  to  assume  was  the  risk  of  the  scope  of  the 
project  exceeding  the  parameters  of  the  project  that  they  agreed  to  build  for  the  price  as 
fixed.  Deloitte  was  always  willing  to  assume  the  technical  risk  that  the  system  they  built 
would  work. 


Appendices 

While  the  references  to  the  United  States  General  Accounting  Office's  Accounting 
and  Information  Management  Division  Suggested  Practices  and  Project  Management  are 
helpful,  there  is  no  need  to  include  those  documents  as  appendices  as  they  provide  an 
unnecessary  level  of  detail  and  specificity.  They  are  appropriate  reference  documents  and 
will  be  used  as  such. 
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Deloitte 
Consulting 


November  16,  2001 

Mr.  James  Tansey 

Director 

Post  Audit  and  Oversight  Bureau 

House  of  Representatives 

Commonwealth  of  Massachusetts 

State  House 

Boston,  MA  02133-1054 

Dear  Mr.  Tansey: 

Thank  you  for  providing  me  with  a  draft  copy  of  the  report  recently  completed  by  the 
Massachusetts  House  of  Representatives  Post  Audit  and  Oversight  Bureau 
concerning  the  Administrative  Office  of  the  Trial  Court  (AOTC)  Information 
Technology  project.  Although  it  appears  that  the  draft  report  is  generally  accurate 
concerning  the  topics  discussed  during  my  interview  on  November  5,  2001,  there  are 
several  areas  where  I  believe  that  some  clarifications  are  necessary.  I  have  attached 
a  document  that  contains  detailed  comments,  referenced  to  specific  portions  of  the 
draft  report.  Please  note  that  my  review  of  your  report  was  somewhat  limited,  and 
based  on  a  quick  review  of  supporting  documents  I  have  available.  The  following 
points  highlight  the  most  significant  areas  where  I  believe  the  draft  needs  to  be 
clarified: 

•     Deloitte's  Work  and  Management  Processes: 

On  Page  27,  the  second  paragraph  contains  an  inaccurate  characterization  of 
Deloitte's  work  and  management  processes.  The  report  states  that  work  was  not 
being  completed  on  time,  or  within  the  cost  range  estimated.  The  report  states  that 
Deloitte  was  not  following  the  PPM  and  implies  that  Deloitte  was  grossly 
underestimating  work  orders.  This  characterization  is  not  correct.  It  is  not 
consistent  with  other  contemporaneous  documentation;  it  is  not  consistent  with 
other  findings  within  your  draft  report;  and  it  is  not  supported  by  the  sole  reference 
cited  in  your  footnotes: 

o    Other  Contemporaneous  Documentation:  During  this  time  period 
there  were  numerous  work  orders  drafted  by  Deloitte  that  followed  the 
PPM  procedures,  and  weekly  status  reports  and  status  meetings  that 
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followed  the  agreed-upon  management  processes.  The  status  reports 
clearly  identify  the  projects  underway,  and  identify  instances  where 
scope  expansions  or  project  decisions  were  causing  deviations  from  the 
original  time  and  cost  estimates.  To  state  that  there  was  a  ".failure  to 
oversee  Deloitte."  and  as  evidence  cite  that  ".work  orders  were  not 
being  completed  on  time."  ignores  the  contradictory  evidence  in  the 
extensive  project  documentation  from  that  time  period,  which  shows  that 
Deloitte  was  closely  tracking  the  progress  of  the  work  and  identifying 
and  attempting  to  mitigate  factors  causing  time  and  cost  increases.  (For 
example,  see  the  Bi-Weekly  Status  Report,  May  14,  1999,  P.  13,  16, 
and  17;  Bi-Weekly  Status  Report,  May  29,  1999,  P.  13  and  15;  PAB 
Meeting  Minutes,  July  12,  1999,  P.  4.)  Similarly,  to  state  ".there  is  no 
evidence  that  any  attempts  were  made  to  utilize  the  PPM  process." 
ignores  many  project  documents  including  the  work  orders  submitted  by 
Deloitte  as  called  for  in  the  PPM,  and  the  status  reports  which  were 
produced  regularly  and  used  actively  during  this  time  period. 

o    Other  Findings  in  Draft  Report:  The  text  on  Page  27  presents  entirely 
different  reasons  for  the  delay  and  cost  increase  in  BasCOT  from  those 
presented  in  footnote  144  on  Page  27,  and  from  the  discussion  on  Page 
43.  The  text  states  that,  as  evidence  of  a  problem  managing  Deloitte, 
work  orders  were  not  being  completed  on  time.  Conversely,  on  Page  43, 
the  report  discusses  BasCOT  development  and  states  "The  delay  and 
cost  was  related  to  the  inability  of  the  AOTC  and  the  departments  of  the 
Trial  Court  to  reach  agreement  on  the  full  and  comprehensive 
functionality  of  the  interim  systems."  This  statement  on  Page  43  is 
supported  by  a  footnote  citing  two  interviews  and  three  project 
documents,  and  Page  43  describes  specific  examples  of  changes 
initiated  by  the  departments  (see  footnote  304,  Page  43).  Footnote  144 
on  Page  27  references  the  interview  with  me  on  November  5,  2001 , 
which  is  consistent  with  the  discussion  on  Page  43  and  alludes  to  the 
reasons  cited  on  Page  43,  but  the  text  itself  does  not  reflect  either  this 
footnote  or  the  more  extensive  discussion  on  Page  43.  The 
characterization  presented  in  the  text  is  not  correct,  and  it  is  not 
consistent  with  your  interview  with  me,  or  with  your  own  findings  in  this 
report. 


o 


Reference  Cited  in  Footnotes:  The  text  on  Page  27  is  supported  by  a 
document  referenced  in  Footnotes  142,  143,  145,  146,  and  147.  This  is 
the  only  document  reference  supporting  the  entire  paragraph,  and  the 
document  is  described  as  "Contract  Renewal  Letter,  July  23,  1999".  This 
is  not  actually  a  letter.  It  is  an  e-mail  from  the  CIO  to  his  in-house 
attorney  asking  for  suggestions  on  provisions  that  could  possibly  be 
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incorporated  into  the  contract  during  the  annual  contract  renewal 
discussions.  Your  paragraph  does  not  represent  the  main  purpose  of 
this  e-mail  accurately.  The  e-mail  identifies  some  possible  situations  ("If 
Deloitte  carelessly  or  incompetently  estimates.";  "If  Deloitte 
misrepresents.")  and  asks  for  suggestions  on  what  provisions  could  be 
incorporated  into  a  contract  to  preclude  such  behavior,  should  it  occur. 
The  e-mail  describes  this  behavior  as  a  "possibility",  yet  your  report  cites 
the  e-mail  as  the  source  document  supporting  your  assertion  that  this 
was  the  situation  at  the  time.  Your  report  takes  several  quotes  out  of 
context  in  ways  that  are  not  consistent  with  the  actual  meaning  of  the 
message.  For  example,  your  report  states,  "The  CIO  complained  that 
Deloitte  and  the  AOTC  needed  to  'agree  on  an  order  mechanism  that 
works  and  then  use  them'",  but  the  relevant  section  of  the  ClO's  e-mail 
states,  "Most  of  my  issues  with  work  orders  and  change  orders  really  do 
not  require  any  change  to  the  contract.  The  issues  require  that  Deloitte 
and  I  agree  on  an  order  mechanism  that  works  and  then  use  them.  I  can 
resolve  these  issues  by  working  with  Deloitte  to  agree  to  mechanisms 
that  work,  to  put  down  on  paper  how  the  mechanisms  operate,  and  then 
to  use  the  mechanisms  diligently  and  consistently."  Your  report  also 
states  that  "Concerns  about  poor  management  and  oversight  were 
exacerbated  by  fears  that  Deloitte  was  'grossly'  underestimating  work 
orders  and  change  orders",  citing  the  e-mail  as  a  reference.  Actually,  the 
quote  referenced  is  not  from  the  CIO  but  from  the  in-house  attorney, 
seeking  clarification  in  response  to  the  ClO's  e-mail,  asking,  "How  often 
has  D&T  grossly  underestimated  the  time  needed  to  complete  a  task?" 
The  e-mail,  read  in  its  entirety,  does  not  support  the  description 
presented  in  your  draft  report,  and  the  fact  that  no  changes  to  address 
these  situations  were  incorporated  or  even  proposed  in  two  subsequent 
contract  renewals  indicates  that  this  e-mail  was  just  a  communication 
between  the  CIO  and  his  attorney  on  possible  situations  that  could 
arise,  and  ways  to  prevent  these  situations. 

In  summary,  the  characterization  presented  on  Page  27  is  not  accurate  and  is  not 
supported  by  the  reference  you  cite.  In  my  view,  the  final  paragraph  on  this  page 
should  be  deleted. 

•     Line  Speed  Requirements  for  BasCOT: 

Page  33  discusses  the  BasCOT  rollout  and  issues  related  to  line  speed  in  courts 
with  56k  lines.  Your  report  cites  a  document  written  by  the  Project  Manager  in 
October  2000  to  conclude  ".an  interim  system  has  been  designed,  developed 
and  partially  deployed  without  any  prior  assessment  as  to  the  speed  of  the 
BasCOT  application  or  the  line  capacities  of  each  court."  Your  report  states  that 
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the  AOTC  and  Deloitte  ".failed  to  perform  an  analysis  of  the  appropriate  line 
speed.."  and  that  "Had  a  technical  consultant  been  in  place.it  would  have  been 
obvious  that  the  existing  network  capacities. would  not  support  full  deployment." 
These  conclusions  are  not  consistent  with  the  facts.  This  section  ignores  a 
substantial  body  of  project  documentation  that  shows  that  the  network  speed  was 
identified  as  an  issue  early  in  the  project  and  revisited  continuously  throughout  the 
BasCOT  projects,  and  the  section  is  not  supported  by  the  sole  document  you 
reference  to  support  these  conclusions: 

o    Project  Documentation  Regarding  Network  Speed:  Deloitte 
produced  a  deliverable  titled  "Network  Architecture"  dated  March  5, 
1998  presenting  recommendations  for  supporting  AOTC  systems.  This 
document  identifies  issues  with  56k  lines,  and  states,  ".the  current 
bandwidth  will  not  support  the  AOTC's  future  plans  for  an  integrated  and 
automated  court  system.  Additional  bandwidth  will  be  required  to 
support  the  new  suite  of  applications."  (Page  8).  The  Network 
Architecture  cites  some  technical  constraints  with  the  NYNEX  (now 
Verizon)  network,  and  recommends,  ".the  AOTC  should  upgrade  all 
remote  courthouse  frame  relay  lines  to  T1  lines."  (Page  9).  Separately,  a 
deliverable  titled  "Proposed  AOTC  Network  Infrastructure",  dated  April 
1,  1999,  and  updating  prior  network  planning  work,  recommends 
upgrading  all  existing  56k  lines  to  384k,  and  upgrading  all  existing  384k 
lines  to  T1  (Page  9).  This  analysis  is  based  on  estimated  traffic  volumes 
and  numbers  of  users  for  BasCOT  and  other  AOTC  applications,  and 
includes  county-by-county  descriptions  of  proposed  changes  and 
estimates  of  costs.  Deloitte's  May  14,  1999  Bi-Weekly  Status  Report 
identifies  response  time  concerns  with  BasCOT  while  waiting  for  a  384k 
line  to  be  installed  (Page  18)  and  the  Project  Advisory  Board  Meeting 
minutes  from  August  9,  1999,  reference  a  discussion  about  the  need  to 
complete  the  Site  Readiness  Assessment  to  determine  if  the  interim 
systems  could  be  rolled  out  on  the  existing  networks  (Pages  18-19). 
These  documents  are  evidence  that  the  network  concerns  were  raised 
early  and  that  continuing  analysis  was  conducted  to  determine  network 
support  requirements.  Substantial  analysis  of  network  constraints  and 
requirements  was  considered  for  both  the  interim  applications  and  the 
anticipated  statewide  application,  contrary  to  the  assertions  contained  in 
your  report. 

o    Reference  Cited  as  Support:  The  text  on  Page  33  is  supported  by  a 
document  referenced  in  Footnotes  212-216,  referenced  as  "Potential 
Drawback  to  BasCOT  Rollout,  Project  Manager  File,  October  2000". 
This  is  the  only  document  cited  to  support  the  text  contained  on  Page 
33.  This  document  is  not  quoted  accurately  and  your  report  does  not 
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disclose  that  this  document  is  the  second  page  of  a  two-page  document, 
with  the  first  page  titled  "Benefits  to  BasCOT  Rollout".  The  "Benefits  to 
BasCOT  Rollout"  page  contains  11  numbered  points,  filling  the  entire 
page,  describing  the  ways  that  the  BasCOT  rollout  will  benefit  the  Trial 
Court,  the  end  users,  the  public  and  the  project  (e.g.,  #1 1 ,  "If  there  is  a 
funding  problem  in  the  future,  BasCOT  will  continue  to  provide  a  viable 
solution"  (emphasis  in  the  original)  "to  automated  case  management"). 
Conversely,  the  second  page,  titled  "Potential  Drawback  to  BasCOT 
Rollout",  consists  of  only  two  sentences  -  the  quote  contained  in  your 
report  is  virtually  the  entire  page.  However,  it  is  not  quoted  accurately. 
The  actual  quote  is  as  follows:  "There  is  a  risk"  (emphasis  in  the 
original)  "that  BasCOT  will  not  be  deployed..",  and  the  second 
sentence  contains  a  parenthetical  expression  which  you  eliminated  in 
your  report:  "It  may  be  cost  prohibitive  or  capacity  prohibitive  (i.e. 
Verizon  or  alternative  vendor  may  not  be  capable  of  meeting  network 
demands)  to  upgrade  the  remaining  courts  to  faster  line  speeds."  Taken 
in  context  and  considering  both  pages,  it  is  clear  that  this  document 
thoughtfully  weighs  the  benefits  and  potential  drawbacks,  as  one  would 
hope  a  diligent  Project  Manager  would,  and  lists  1 1  potential  benefits 
and  only  one  risk  of  a  potential  drawback  -  the  emphasis  in  the  original 
is  important.  Moreover,  the  parenthetical  reference  to  Verizon  technical 
constraints  points  back  to  the  original  1998  analysis  referenced  above, 
showing  that  the  network  considerations  were  an  important,  and 
continuing  topic  of  analysis  and  management  concern  from  the  earliest 
days  of  the  project.  Furthermore,  although  this  section  cites  my 
interview  as  a  reference,  I  was  never  asked  about  this  subject  during  my 
interview,  and  I  would  have  readily  clarified  any  misunderstanding 
surrounding  this  topic.  This  document  does  not  support  the  assertions 
made  in  the  draft  report. 

In  summary,  the  characterization  presented  on  Page  33  is  contradicted  by  project 
documentation  and  is  not  supported  by  the  document  you  reference.  In  my  view,  the 
text  in  these  paragraphs  should  be  deleted  or  substantially  revised. 

•     Termination  of  Contract  Negotiations: 

Page  38  discusses  the  termination  of  contract  negotiations  between  the  AOTC 
and  Deloitte,  and  references  the  1999  "Contract  Renewal  Letter",  as  you  referred 
to  the  e-mail  cited  above.  This  section  incorrectly  characterizes  the  topics  of 
negotiations  as  "long  standing  problems"  that  had  been  ".focal  points  of 
contention  during  the  life  of  the  integrator  contract."  As  discussed  earlier  in  this 
document,  the  e-mail  you  cite  as  a  reference  was  a  communication  from  the  CIO 
to  his  attorney  asking  for  suggestions.  In  addition,  your  report  does  not  incorporate 
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the  discussion  from  my  November  5,  2001  interview  with  the  Bureau,  when  I  said 
that  the  issue  that  led  to  the  termination  of  negotiations  was  a  difference  in  views 
on  how  to  allocate  the  risk  of  managing  scope.  The  other  issues  you  list  were 
topics  for  negotiations,  but  were  not  deal-breakers.  Certainly,  if  issues  you  list 
such  as  limitation  of  liability,  damages,  ultimate  ownership  of  software,  and  the 
appropriate  technical  and  application  architecture,  had  been  "focal  points  of 
contention",  and  if  issues  involving  testing,  acceptance  of  deliverables,  project 
structure  and  payment  had  been  "long  standing  problems",  the  AOTC  would  have 
at  least  proposed  rectifying  some  of  these  issues  during  the  repeated  contract 
renewals  and  work  order  approvals  that  occurred  between  July,  1999  and  April 
2001.  It  is  unfortunate  that  our  negotiations  were  not  successful,  but  these  issues 
are  typical  of  the  issues  that  are  dealt  with  in  every  carefully  negotiated  fixed  price 
system  development  contract.  It  is  important  to  note  that  subsequent  to  the 
termination  of  negotiations  our  System  Integration  contract  was  continued  by  the 
AOTC,  and  we  are  still  assisting  the  AOTC  with  the  project  today,  more  than  six 
months  after  the  end  of  the  negotiations. 

This  paragraph  is  inaccurate  and  is  not  supported  by  the  document  referenced.  In 
my  view,  this  paragraph  should  be  substantially  revised. 


I  want  to  thank  you  for  the  opportunity  to  review  the  draft  report  and  provide  you  with 
my  comments,  and  for  the  courtesy  extended  to  me  during  my  interview  with  the 
Bureau.  Although  the  references  from  my  November  5th  interview  have  been 
accurately  captured  for  the  most  part,  the  issues  described  above  and  outlined  in  the 
attached  document,  in  my  opinion,  are  significant  and  should  be  addressed.  I  hope 
that  you  will  consider  these  comments,  and  correct  what  I  believe  to  be  oversights 
that  are  not  essential  to  your  main  points. 

If  you  have  any  question  about  this  letter  or  my  comments,  please  feel  free  to  call  me 
at  (617)  850  2010. 


Sincerely, 


Paul  Cunningham 
Partner 

Enclosure 


